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Preface 



The last few years have witnessed the publication of a large 
amount of material on programed instruction and teaching ma- 
chines. Much of this material bears testimony to the increasing 
maturity of this new art and to the growing sophistication of pro- 
gram writers. The question "Will it work?" has been replaced 
by the question "How can it be used most efficiently?' 7 and by 
detailed inquiries into cost, preparation time, available devices, 
and other utilization data. 

This book has been designed to answer these inquiries. Our 
contributors have, for the most part, chosen not to emphasize 
theoretical questions but, rather, to focus on practical, imple- 
mentation data. Descriptions of program users, sources of pro- 
grams, feasibility of internal production, difficulties of utiliza- 
tion, and economic factors form the central topics of discussion, 
but the applicability of devices, or even computer-based teaching 
systems, is also covered in some detail. 

In addition to thanking the authors of these articles, we would 
also like to thank John Annett of Sheffield College and William 
Barton Marsh of the agency of that name for their lucid analy- 
ses of the industrial utilization of programed instruction. We 
would especially like to thank Bernard Basescu of The Center 
for Programed Instruction for his helpful comments and invalu- 
able assistance in the preparation of this book. We would also 
like to express our gratitude to The Center for Programed In- 
struction for providing us with a home from which we might re- 
view the applications of programed instruction. 

We really must remark on the unselfish cooperation of our 
authors. Half of them spell it "programmed." We appreciate 
their willingness to adopt the spelling with one "m" for the 
sake of uniformity in this collection. 

New York, New York Stuart Margulies 

Februrary, 1962 Lewis D. Eigen 
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SECTION 1. Introduction 



Most of the articles in this book focus on specific aspects of 
program construction. They are concerned with detailed ac- 
counts of military utilization, with economic appraisals, or with 
evaluations of available devices. In these articles, the reader 
generally is expected to have looked at several programs and 
to be familiar with such terms as "reinforcement" and "frame. 7 

Those seeking a general introduction to programed instruc- 
tion and those interested in obtaining an overview of this bur- 
geoning new field will find this first section both useful and 
interesting. Cook and Mechner in their article, and Goldberg 
in his, provide accounts of the development of interest in this 
area, describe the characteristics of programed material, and 
outline the theoretical considerations involved in program con- 
struction. They acquaint the reader with the terminology of the 
field and introduce him to the problems with which later sec- 
tions are concerned. 

The reader who has not yet had the opportunity to examine 
programed material might find it helpful to read Appendices E 
through J in conjunction with this section. 



Fundamentals of Programed Instruction 



By DONALD COOK and FRANCIS MECHNER 
Basic Systems, Incorporated 



The term "programed instruction" is perhaps the most widely 
used designation for a new and rapidly developing field that may 
be described as the application of the science of learning to the 
tasks of training and educating. The methods of this new field 
rest on a number of principles of learning that have been widely 
known but never before brought into simultaneous operation in a 
systematic way. Research in testing had shown that when tests 
are scored automatically, learning is improved if students can 
know their scores immediately after each question. Further, ap- 
plied research in military and industrial training had paved the 
way for the use of automatic devices to individualize training. 

But the systematic foundations of programed instruction 
awaited a workable account of learning that could be applied to 
design in a technology of instruction. Primary credit for this 
step must go to experimental psychologists and, in particular, 
to B. F. Skinner of Harvard University. Several decades of study 
of the learning process under carefully controlled conditions en- 
abled Skinner and his colleagues to establish important princi- 
ples that apply to all learning. The first of a series of papers 
describing the practical use of these principles was published 
in 1954, and, since that date, the field of programed instruction 
has made rapid strides of great and continuing promise. 

At the heart of the method is the conception of human behavior 
as an orderly process that can be studied and understood in de- 
tail by the methods of natural science. Such a conception led to 
intensive laboratory explorations in the development and mainte- 
nance of increasingly complex forms of behavior. It soon became 
clear that it was possible to design complete " learning environ- 
ments" in which behavior could be brought to a desired final 
state through a series of graduated stages. In each stage, the 
key process is the arrangement of a situation such that the be- 
havior then occurring produces consequences or effects that will 
bring the learner to the next stage. The design of these stages 
. and effects is at the heart of all approaches to programed in- 
struction. 

Intensive research along the lines indicated by Skinner has 
been widely supported by the United States Air Force and the 
Office of Naval Research, the Ford and Carnegie Foundations, 
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the United States Office of Education, and other funding agencies. 
Experimental projects have been carried out in several hundred 
schools, universities, research institutes, and industrial inter- 
prises, often in collaboration with each other. In case after case, 
significant gains in learning speed and effectiveness have been 
achieved by instructional programing. Methods for more effect- 
ive programing are being discovered and refined, and a number 
of journals and newsletters have been established that collect, 
analyze, and disseminate these findings. 

THE ELEMENTS OF PROGRAMED INSTRUCTION 

The elements combined by instructional programing to pro- 
duce optimal learning are as follows: 

a. Active response by the student. 

b. Small steps in which careful control of stimuli produces 
gradual increments in mastery of the subject. 

c. Immediate feedback for each response. 

d. Self -pacing, or individualization of the rate at which the 
learner masters the material. 

e. Low error rate for the individual learner, as a consequence 
of the effective operation of the first four principles. 

It can be seen that the above principles incorporate both teach- 
ing methods directed at the individual learner and consequences 
for the administration of instruction to large groups. A program 
that incorporates these principles requires the learner's active 
participation at each step. He must carry out an instruction, fill 
in a blank in a sentence, answer a question, solve a problem, or 
complete a diagram. As soon as the learner has made the re- 
sponse, the program presents the correct answer. E the response 
was right, the immediate confirmation reinforces or rewards the 
response. If the learner was wrong, his error is corrected be- 
fore he builds new knowledge on a shaky foundation. 

In this way, errors and misunderstandings do not accumulate. 
Each new step the learner takes is small, building naturally on 
the material just mastered. In a good program, the curriculum 
is designed in a rational and cumulative manner, with maximum 
utilization of the inherent organization of the subject. The stu- 
dent is neither held back by slower students nor penalized if 
he moves slowly through the program. As a consequence he 
makes very few errors ideally, none at all. 

Active Response 

Passive exposure to instructional material does not ensure 
that the learner responds to it and so incorporates the material 
into his own behavior. Listening and reading might result from 
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good instruction, but they alone cannot be guaranteed to produce 
it. Writing, speaking, selecting, and matching, however, involve 
the learner actively at frequent intervals in accordance with the 
guiding principle of learning theory that a response must occur 
before it can be reinforced or rewarded and so become estab- 
lished. To learn is to be able to do, and to understand is to be 
able to explain. 

The response principle is of further importance in that it pro- 
vides a basis for the evolution of instructional programs. When 
a record is made of the learner's behavior, the record can be 
studied to discover where the program is weak, and the program 
can be revised to eliminate errors. 

Small Steps 

Material is presented one step at a time so that it can be un- 
derstood easily. The increment of difficulty between any two 
steps is typically narrow, so that the total sequence ensures full 
preparation for more complex and difficult material. Although 
the final behavior demanded might be quite complex, the learner 
is confident of his progress at every point. 

In the design of an ordered sequence of such steps, the stimu- 
lus support for a given response might be strong at first and then 
gradually withdrawn over a series of steps, with the result that 
a sophisticated or complex response can finally occur on its own. 
This result is possible because each frame (the basic unit of in- 
structional material) presents material that is carefully arranged 
in terms of what the learner knows already: the learner, in mas- 
tering such a step, need not refer back to earlier material or look 
ahead to later material. 

Immediate Feedback 

When each response gets quick tutorial appraisal, learning is 
enhanced. Knowing the correct answer rewards the behavior, 
gives the learner confidence, and encourages retention. This ef- 
fect, of prime importance in instructional design, psychologists 
term 6< reinforcement. 7 ' It has been shown that, if the circum- 
stances are well-prepared, a single reinforced occurence of be- 
havior can establish that behavior in full strength. Although 
knowledge of results is the most usual form of reinforcement in 
programed instruction, other varieties are possible. Advancing 
to the next frame of the program can be reinforcing in its own 
right because it indicates mastery of the previous step and an 
opportunity to enlarge a skill. The number of frames success- 
fully completed can be tallied, with some extrinsic incentive 
made contingent upon this total. 
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Self-Pacing 

Classroom teaching, whether by conventional lecture or by 
novel and exciting stimulus media, invariably addresses a hypo- 
thetical average student who might, in fact, not exist. The rapid 
learner is held back, and the slower learner who might, none- 
theless be a good student is dragged forward too quickly. Few 
students have a chance to respond in any given session, and the 
teacher might favor those who are most likely to give gratifying 
answers. In contrast, programed instruction is learner-centered, 
and encourages each student to work at his own best rate. Such 
individualization permits the student to pause for reflection with- 
out penalty. Working at a comfortable speed, the learner is, at 
the same time, given incentives for steady advance through con- 
stant reinforcement. The total instructional cycle increment, 
question, response, feedback can take place dozens, or even 
hundreds or more, times an hour, with teaching and testing 
blended into one seamless process. 

Low Error Rate 

Research in instructional programing demonstrates the feasi- 
bility of revising programs during their development so that 
errors produced by learners can be reduced to a minimum. In- 
dividual people differ, but if all run into trouble at the same point 
in a program, a revision of the program is clearly indicated. Re- 
visions always take their departure from frames that generate 
high rates of error. But it is not necessarily those frames that 
are revised because an error at a given frame might indicate a 
weakness earlier in the program. The process of revision almost 
always lengthens a program. Even a single such revision might 
not only cut the error rate in half but might increase the speed 
at which the revised (and longer) program is completed. 

Research on revising programs represents a significant ad- 
vance in learning theory, and indicate that it is possible to learn 
to avoid errors without actually having to make them. " Bright' ' 
students characteristically make few errors with no loss in their 
capacity to learn. Error-free learning is not only simpler, but 
its effects improve morale, motivation and retention. 

Errors in learning may be likened to the cost of a product. 
Programed instruction, as a method, produces the product- 
learning at a lower cost than has heretofore been possible. 
Naturally, measured learning achievement the profit derived 
from the product remains the final arbiter. Precautions, there- 
fore, are necessary to guard against poorly-prepared programs 
that produce few errors but do not assure much real learning. A 
good program will be mastered with a low error rate, but a low 
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rate of errors does not automatically signify a good program. 
The program also must teach the material. 

THE DEVELOPMENT OF INSTRUCTIONAL PROGRAMS 

An instructional program is a sequence of items (often called 
"frames") throughout which the above principles are applied. 
A successful program "shapes' 7 the student toward the desired 
knowledge or skill by requiring him to complete a series of 
questions of gradually increasing complexity. At the beginning 
of the program the student makes simple responses to simple 
questions. As the program advances the required level of be- 
havior gradually advances until the student is finally giving 
sophisticated answers to complex questions. These answers 
might involve large-scale analyses and integration of materials, 
the writing of short essays, the solution of difficult problems, or 
the drawing of elaborate diagrams. A program can accomplish 
this by applying certain principles of verbal learning and the 
theory of concept formation. The better known techniques for 
establishing verbal knowledge include cueing, interweaving of 
principles and examples, fading or vanishing of stimuli, con- 
current development of related concepts and, concept formation 
by induction. 

Design of a Program 

The first step in planning an instructional program is to spec- 
ify the knowledge or skill that is to be imparted to the student. 
This knowledge or skill is called the terminal behavior because 
it is the behavior the student should possess after he has com- 
pleted the program. The terminal behavior of a geometry pro- 
gram, for example, might be specified as the ability to score 
90% or better on the New York State Regents examination in 
geometry. In general, the specification of the terminal behavior 
must refer to an examination (or alternate forms thereof) or to 
tasks that the trainee must be able to perform as well as to the 
level of performance to be achieved. 

It is important to realize that a course outline is not a speci- 
fication of terminal knowledge. For example, it is not sufficient 
in a course in electronics to specify an "understanding of diodes* 
because it is not clear what kind of knowledge is being specified* 
Such a specification does not distinguish between the following 
alternatives: 

a. What is a diode? 

b. What does a diode do in an electrical circuit? 

c. Name four different types of diodes, explain how each one 
works, draw its characteristics, and explain for what type of 
application each one is suitable. 
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The above alternatives represent three vastly different degrees 
of mastery, and it is vital for the programer to know the precise 
level required. In short, the behavioral outcome to be produced 
must be made explicit; a set of representative problems that sam- 
ple the behavior of the trainee must be analyzed to produce a des- 
cription of the capabilities that must be generated by the program. 

Program Construction 

Once the required terminal behavior has been specified, the 
program can be constructed. In some cases, a detailed outline 
or flow chart of the program might be required in which the 
topical and behavioral objectives of the program are set forth 
in sequence. The actual creation of program frames is accom- 
plished by programers who are specially trained in the neces- 
sary behavioral principles. The training of programers is a com- 
plex process. It can be greatly enhanced through the tutelage of 
a teacher already skilled in programing theory, but the expe- 
rience of testing frames on actual subjects is indispensable. 

Close collaboration or cross -disciplinary training between in- 
dividuals with subject-matter skills and those with programing 
skills might be required. This can be achieved in several ways, 
depending on the skills involved. The manner in which frames 
are prepared depends on the nature of the behavior to be trans- 
mitted. Verbal skills, involving paper -and-pencil knowledge, 
can be programed on a series of index cards, with one frame on 
each card. Perceptual skills might require considerable art 
work or other stimulus material. Skills based on eye-hand co- 
ordination might preclude reading and, therefore, require an 
audio input through headphones. Similarly, feedback might come 
in printed or visual form, it might be spoken from tapes, or it 
might consist of signal patterns from the apparatus being mas- 
tered. In all cases, programs begin relatively simply, assuming 
only the behavior that members of the subject population will 
bring to the program; and they end with a series of criterion 
frames, mastery of which is equivalent to possessing the full 
range of terminal behavior to be produced by the program. 

Validation of Programs: Testing and Revision 

Program validation is an empirical process. While it is being 
developed, the program is repeatedly tested on typical members 
of the subject population who take examinations on the material 
they are learning. If errors concentrate too heavily at certain 
points in the program, the program must be revised. Revisions 
thus are empirically checked and repeated until the final effect 
is attained: a reliable production of the terminal behavior as 
measured by the external criterion test and a satisfactorily low 
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error rate as an internal criterion of good programing. As ad- 
vances are made in the behavioral principles that underlie pro- 
graming, the number of revisions required to produce a good 
program can be expected to decrease. The final test of a program 
might be a field test in which a group of students who are actual 
members of the subject population complete the program under 
field conditions and then take a final examination. Satisfactory 
performance on this examination constitutes the final validation 
of the program. 

Format of Instructional Programs 

The format of a program depends on the nature of the behavior 
it must generate. A common form of verbal programing is now 
available in the programed textbook. Resembling a book, the 
programed text contains pages that present frames and answers 
in a certain order. In one version, the frames progress verti- 
cally down the page with the answers in adjacent columns dis- 
placed downwards by one step. In such a format, a slider is of- 
ten used to mask answers and subsequent frames. Another wide- 
spread format presents frames and answers on successive pages 
so that, after a frame has been answered, the page must be turned 
to expose the confirming correct answer. The learner works 
through the entire book by taking the frames at the top level of 
each page and then returning to the first page and working through 
the book again at the second level, and so on for anywhere from 
three to six levels, depending on frame size. This format entails 
a good deal of page turning but has the advantage that a given 
frame is not surrounded by neighboring frames in the instruc- 
tional sequence, and consequent contamination from undesired 
hints and prompts is avoided. 

Format and Pedagogical Problems 

In one version of programing, each frame presents a paragraph 
of instruction that is followed by a multiple choice question. 
Choice of the correct alternative advances the student to the next 
frame, but choice of an incorrect alternative sends him to a reme- 
dial frame, after which he returns to the main frame and chooses 
again. Psychologists have been critical of such a format on peda- 
gogical grounds. Selecting an answer is a more passive process 
than actually constructing it, a fact that might affect the learner's 
ability to recall the response later. 

The basis of the selection might be the elimination of wrong 
alternatives rather than the strengthening of the correct one. A 
lucky guess might advance a learner who could well profit from 
the clarifying material in remedial frames. Studying wrong 
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alternatives that at least must be plausible enough to be worthy 
of inclusion might tear down the very learning that has just been 
built up before it has been reinforced. The learner might later 
remember wrong alternatives he has studied, even though he was 
fortunate enough to reject them at the time. The use of multiple- 
choice answers might discourage program designers from test- 
ing programs, analyzing errors, and revising programs so that 
each learner benefits from every step. 

Such points, though persuasive, canndt be regarded as settled. 
Research to date, which has characteristically employed brief 
programs and has not tested for long-term retention, has not 
shown the significant differences that would be of importance in 
lengthy programs. These and other issues can be expected to 
occupy the talents of research psychologists now being attracted 
to the field in increasing numbers. 

The Instructional Frame 

The basic unit of programed instruction is the instructional 
frame. It presents the increment of instruction and calls for the 
response from the learner. After this response receives evalua- 
tive feedbacktypically, presentation of the correct answer a 
new frame begins. Thus the cumulative effect of a sequence of 
instructional cycles depends on the design of a large number of 
frames in the proper order. The topic of frame design and se- 
quencing is receiving increasing attention as the theory of pro- 
graming develops. The number of aspects to be considered is 
as rich as behavior itself , and only a few points will be made 
here. 

The Initial Repertoire. The behavior brought to a program by 
a learner obviously conditions what can be done within the pro- 
gram. In a paper-and-pencil program, he must be able to read. 
Less trivially, the nature of the verbal connections existing 
" within him" is of great importance. A frame such as "If auto- 
mobile means 'self moving/ then. might mean 'self- 

instruction/ " obviously depends on a repertoire of pre-existing 
verbal fragments and combination rules . The general knowledge 
the learner can be assumed to have will affect strongly the use 
that can be made of examples in building up new principles. 

Control of Cueing. Responses must be made for the right rea- 
sons. But when first called for, a correct response might be 
made easy by merely requesting that it be copied, by giving syn- 
onyms, by prompting with words having similar roots, by prompt- 
ing with rhyming sounds, by giving a part of the word, etc. These 
initial cues are then gradually withdrawn, and the strategy by 
which this is done, as well as the selection of cueing methods in 
the first place, is a key aspect of frame design. 
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Over-cueing. Over-cueing is said to occur when the right re- 
sponse is made too easy, or when it is ''given away/' with a loss 
in opportunity to strengthen it correctly. To write " There are 
nine planets. The Earth is a planet. Therefore it is one of the 

planets/' over-cues the response in that the blank 

is filled with exactly the same word as preceded "planets" ear- 
lier in the frame. A foreigner knowing no English might fill the 
blank correctly through a shrewd guess because of the cueing 
provided by the sequential redundancy. This frame might better 
name all the planets, then ask for their number. The learner 
who can count will then construct an active response. Beginning 
programers are prone to over-cueing, as are most teachers who 
are anxious to hear some correct answers. 

Relevance. It probably is best to include in each frame only 
the material that is required to construct the answer. This prac- 
tice helps ensure that the frame will be attended to. The tempta- 
tion to "sneak in" incidental material results in an over-ambi- 
tious program, the effects of which will be uncertain. 

Review. Material must be reviewed at appropriate intervals. 
Unlike the memory drum, the frames of an instructional program 
are not designed to be taken repeatedly; therefore review must be 
"seeded in" throughout the frames. To avoid rote review, the 
examples or context illustrating the point can be varied so that 
new material is integrated with old. In good programing, such 
review advances the work of the program. 

Panels. A sequence of frames might refer to a separate panel 
a single display that the learner uses throughout the sequence. 
The main function of the panel might be that of auxiliary stimula- 
tion, as when the frames ask a series of questions about a map; 
the panel might serve as reference material, as when the frames 
teach the learner how to use a table of mathematical functions; 
or the panel might be an arena for behavior, as when a series of 
frames instructs in a series of arithmetic steps that are "col- 
lected" on the panel so that a new generalization can emerge in- 
ductively. 

Branching, This term refers, not to an aspect of any single 
frame, but to options or choices among sequences of frames. 
Many variations of branching can be used, depending on the prob- 
lem. "Express stops" can be employed where a student is 
branched ahead to a later stage of the program if he fills out a 
diagnostic frame correctly. Branching can lead into remedial 
routines or into optional enrichment of a subject. Programs have 
been designed in fast and slow tracks with learners shifting as a 
function of their error rates on a given track. Still other varia- 
tions of branching are sure to develop in the future. The design 
of useful branching strategies designed to accommodate individual 
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differences will be most sound if first consideration always is 
given to the interaction between the individual learner and the 
program. 

TEACHING MACHINES 

The term "teaching machine" is best reserved for devices 
that house, display, and present instructional programs of the 
sort defined and described earlier. Thus, the machine not only 
displays material, it has provisions for suitable responses on 
the part of the student and provides tutorial feedback. By these 
criteria, audio-visual aides, as well as simulators and other de- 
vices, although they might be machines used in teaching, do not 
qualify as teaching machines. 

General-Purpose Machines 

Teaching machines now available that have the capability of 
presenting instructional programs can be simply described. 
Such machines are primarily intended for paper-and-pencil pro- 
grams. They house the program on rolls of paper, flat sheets, 
or fan -folded sheets. A display aperture exposes each instruc- 
tional frame, and a second aperture provides a space for the 
response. Feedback is given when the program is advanced- 
through actuation of a lever, knob, or buttonand the correct 
answer comes to view. It is probably best that the frame and 
the learner's answer both remain in view when the feedback 
is presented. For certain programs that require the building 
of a new response repertoire (as in drawing) , or when teaching 
young children who cannot verbally encode their own responses, 
this feature of machines can provide a genuine pedagogical ad- 
vantage. When frame, response, and correct answer are all in 
view together, the learner's response can be made to advance 
under a window to prevent erasure or " cheating." 

Teaching machines may or may not score responses and tally 
errors. At present this attractive degree of automation is possi- 
ble only with certain machines or with programs in which re- 
sponses can be selected (rather than constructed) and the choice 
indicated by pressing a button. With constructed-response pro- 
grams, the evaluation of the answer and a tally of errors must 
be left to the student a requirement that is not unreasonable 
for older students. The day may come when optical scanning 
(or its equivalent) , together with mechanized semantic analysis 
to assess fine equivalence of utterances, will make these ma- 
chines fully automated. 

Regardless of the manner in which these technical problems 
are solved, it appears that a permanent place will be assured 
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the programed textbook. It is cheap, flexible, portable, and does 
not require a large capital investment. Free from the restric- 
tions of a fixed-size aperture for the answer space, programed 
texts can accommodate a wide variety of responses. Where an- 
swer books are produced separately, the program itself is not con- 
sumed and can be reused. For instruction in primarily verbal 
material with students whose fundamental verbal behavior has 
already been established, the programed text seems to have 
many advantages and no defects. 

Cheating with the programed textbook format might be a prob- 
lem with young children or students who do not yet understand 
that the program is intended to teach rather than test. In such 
cases, machine presentation often is justified. In the case of 
older students, the problem can be solved by demonstrating that 
the program is designed to build behavior, not to test it. The 
periodic administration of actual examinations is an effective 
way of conveying this distinction. Together with the intrinsic 
factor that cheating deprives a student of reinforcements or 
rewards in mastering a program, the use of tests should elim- 
inate the motivation to cheat while taking a program. 

Special-Purpose Machines 

Special-purpose devices have been developed that are indis- 
pensable for teaching certain skills. Among these skills are 
audio -manipulative coordination tasks, tasks where speed and 
motor accuracy are essential, and tasks where the eventual 
work environment must be simulated. Other machines have 
been developed for special training in areas such as pre -verbal 
matching of form and color, matching of rhythmic patterns, etc. 
In each of these cases, the unique requirements of stimulus pre- 
sentation and response topography dictate the design of machines 
with special features. 

PROGRAMED INSTRUCTION IN BUSINESS AND INDUSTRY 

As the training needs of modern business and industry con- 
tinue to mount, programed instruction can be expected to play 
an extraordinarily useful role. The internal training of skilled 
operators is a classical problem of long standing. The increased 
instructional effectiveness that programing will bring might well 
alter the strategic role of selection tests and broaden the base of 
recruitment to a wider population range. Allocation strategies for 
already selected men similarly might become more flexible. 

In developing training courses, existing training manuals might 
be helpful in some instances especially with well-organized tech- 
nical material that is primarily verbal in nature (electrical wiring 
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and blueprint reading are good examples). In other cases, exist- 
ing training manuals might have to be scrapped and the terminal- 
behavior requirements developed from scratch. 

The nature of the terminal behavior might well depend on 
critical features of the post-training work environment an 
increasingly important subject in industrial design. The princi- 
ples of man-machine-system design call for the incorporation 
of automatic, on-line feedback signals, which serve to motivate, 
guide, clock, and count aspects of production. Thus, the magni- 
tude of the task assigned to training is reduced. These facts 
suggest that industrial-training-program planning should emerge 
as a collaborative effort between industrial and programing enter- 
prisesan effort extending in both directions beyond the training 
phase per se. 

In the case of sales training, programed instruction has a spe- 
cial potential in the training of widely dispersed personnel. A 
good example is that of pharmaceutical detail men who must re- 
ceive continuous on-the-job training to keep abreastat a rather 
technical level of a continuous flow of new products. The prom- 
ise of programed instruction in the reduction of travel time, 
cost, and interruption of sales activities is obviously great. 

As new technical products proliferate, business and industry 
finds itself taking on the job of educating the consumer. He must 
be trained to utilize complex products if he is going to buy them. 
In the computer industry, where the consumer must master a 
new problem-oriented language, plans must be made for instruct- 
ing consumers in that language before the hardware that will 
obey the language has been assembled. More prosaically, such 
firms often face the task of training operators sent by customer 
companies in the use of equipment that has been purchased. Here 
the educator exerts almost no control over the qualifications of 
his students. Clearly, improved instructional methods will be of 
great help in such situations. 

Management itself is not exempt from the impact of the train- 
ing revolution. Increasing demands are made to keep abreast 
of new processes and products. Managerial techniques them- 
selves become increasingly complex. Often the operating des- 
criptions of cost-accounting, production-control, and other ration- 
alizing procedures go unread or unused because they are not taught 
effectively. Management games, simulation training, and other 
techniques for generating company strategy in an age of minimax 
and probability themselves pose the need for a "strategy of ab- 
sorption." As companies diversify, decentralize, and process a 
rapidly altering set of products and services, the need arises for 
an informational nervous system a communication network to 
maintain the corporate identity that to an increasing degree, does 
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not inhere in any one man or group. Electronic communications 
now can provide the required annihilation of space and time and 
the effective storage and processing of data, but the final mem- 
brane of the skilled behavior of men remains as a barrier and 
requires a technology of its own to penetrate. 

In view of all these trends, it might well be that management, 
increasingly aware that its "hard" product is enshrouded in 
issues that center around the training of men, might bring train- 
ing policy closer in to the round table of executive decision. 
Training costs and processes should be brought out from under 
the dark corners of i{ overhead," reject rates, and other hidden 
and obscure budget categories. In this process, programed in- 
struction offers a technique of increased effectiveness that will, 
at the same time, allow training costs to be rationalized effec- 
tively, evaluated accurately, and amortized in a sensible manner. 
That is the final prospect of this rapidly developing field for in- 
dustry today. 



An Introduction to Programed Instruction 



By IRVING A. GOLDBERG 

Automated Teaching Systems, Incorporated 



This brief question-and -answer article has been prepared to 
answer the various questions that have been asked about pro- 
gramed instruction and to summarize its underlying principles 
and cJiaracteristics. 

Question: What do the terms "programed instruction 39 and 
"teaching machines" mean? 

Answer: The terms '* programed instruction" and f< teaching 
machine ?? refer to a revolutionary new development in educa- 
tional technology that has been gathering momentum during the 
past six years. This new development is based on principles of 
learning theory that derive from modern experimental psychol- 
ogy. Its foundation lies in control over the interaction between 
the learner and the material to be learned. 

Question: What, briefly, is the history of programed instruction? 

Answer: Although the principles underlying programed instruc- 
tion can be traced back to ancient Greece and the Socratic method 
of tutorial education, modern developments may be said to date 
from the work of Dr. S. L. Pressey at Ohio State University in 
the early 1920>s. In 1926, Dr. Pressey developed a testing device 
that is considered as the forerunner of present-day efforts in the 
automation of education. Unfortunately, Dr. Pressey' s early work 
met with little public support. 

The new educational technique was virtually forgotten until 
1954, when a resurgence of interest was initiated by the work of 
Prof. B. F. Skinner and his associates in the psychological lab- 
oratories of Harvard University. Since then, programed instruc- 
tion has received broad recognition and support. Numerous edu- 
cational, military, and industrial organizations have undertaken 
pilot programs to test the effectiveness of the new method, and 
various commercial ventures have been directed to the develop- 
ment and marketing of vocational and academic programs of in- 
struction and many types of automatic teaching devices. 
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Question: What are the principal characteristics of programed 
instruction? 

Answer: The instructional material is developed by careful 
analysis of the behavior to be taught. All information, actions 
and decisions required for successful performance by the 
learner are carefully specified. 

The subject-matter is broken down into a series of short in- 
structional steps called frames. These are organized in a log- 
ical step-by-step sequence that leads the learner from his pre- 
program behavior to the desired terminal behavior. Each frame 
provides one or more informational statements or illustrations 
and incorporates one or more questions to which the learner 
must make an active (verbal, written, or physical) response. 
Following his response to each question, the learner is immedi- 
ately shown the correct response to compare with his own. 

The learning session is self -paced by the learner. The pres- 
ence of a teacher is not required, although, in many instances, 
it may be desirable. 

Question: What are the psychological principles on whicJi pro- 
gramed instruction is based? 

Answer: Programed instruction is based on the principle of 
"reinforcement" providing a reward for desired behavior so 
that the behavior tends to re-occur. Desired learning is auto- 
matically "reinforced" with programed instruction by immedi- 
ate confirmation of each correct response that the learner makes. 
The learner's self -awareness of successfully responding is in- 
herently rewarding. Programed instruction structures the learn- 
ing situation so that the learner is able to progress with mini- 
mum errors from one learning step to the next. The learner's 
knowledge of his capability to progress successfully is another 
source of reinforcement and, thereby, serves to enhance the 
effectiveness of the learning process. 

Question: What is the instructional cycle with programed in- 
struction? 

Answer: The cycle of instruction consists of the following steps: 

1. The learner reads a small unit of instructional material 
(an instruction, a fill -in -the -blank question, a multiple -choice 
question, an essay question, or an incomplete diagram). 

2. He makes a response by carrying out the instruction, filling 
in the blank, making a selection from the multiple choice, an- 
swering yes~or-no, or completing the diagram. 

3. He is shown the correct answer and advances to the diagram. 
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Question: How is a program of instruction presented to the 
learner? 

Answer: Programs of instruction are most commonly presented 
to the learner either in book format or in one of several other 
formats that are appropriate for use with teaching machines. 
Book -format programs are prepared either in printed-and-bound 
form, or in the form of mimeographed work sheets. Programs 
for machine presentation are prepared in film form (for projec- 
tion) or printed on paper rolls or tapes or sheets (for direct read- 
ing). Several very complex machines include the instructional 
program as an integral part of the device; the presentation and 
sequence of the program is controlled by changes in the state of 
electrical circuitry or mechanical components. 

The physical arrangement of the instructional material within 
each program is accomplished in several ways. Typical arrange- 
ments present the instructional items in numerical order, with 
the answer to each question provided beside it or on the follow- 
ing page. 

Depending on the particular book -form or machine program 
that is used, the learner's response can be made directly on the 
presentation medium itself, on an answer -book or answer -tape 
that is provided separately from the presentation medium, or 
by means of input controls provided for response purposes. With 
book -format programs, a means is usually included for the learner 
to mask the answer to each question until after he has made his 
own response. With machine programs, masking of the correct 
answers is accomplished by the device itself; the exposure of 
each answer is either automatic immediately after the learner 's 
response or is under the mechanical control of the learner. 

Question: What are the principal characteristics of a teaching 
machine? 

Answer: Numerous types of teaching machines have been devel- 
oped. They vary from very simple devices priced at under $5 to 
very complex devices priced up to as much as $20,000. In general, 
all types of machines, regardless of price, incorporate the follow- 
ing characteristics: 

a. A program of instruction is exposed, one frame at a time, 
to a learner, who must respond in some way to each frame be- 
fore the program is advanced to the next one. 

b. After each response by the learner, the machine exposes 
the correct answer. 

c. The pace of exposure of the instructional item is controlled 
by the learner; no teacher need be present. 

It is evident that, in general, the teaching machine merely 
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serves as a convenient means for the presentation of programed 
instruction. As such, the machine itself is of secondary import- 
ance to the learning process. But it performs several important 
functions: it permits automatic control over the exposure of the 
instructional material and thereby prevents cheating; it elimi- 
nates the need for complicated instructions; it provides a simple 
means for recording and scoring the learner's behavior; and it 
can aid in maintaining the learner's interest becauseof its " hard- 
ware' ' characteristics. 

Question: What are the steps involved in the development of 
an industrial training program ivitJi programed 
instruction? 

Answer: The development of a programed-instruction course 
for industrial application is accomplished by the following steps: 

1. Task Analysis, The programing company undertakes an 
analysis of the tasks, skills, and procedures that are required 
for satisfactory job performance. Specific training requirements 
are defined in consultation with responsible personnel of the client. 

2. Content Developments. Subject-matter is developed in terms 
of a behavioral definition of the component skills and informations 
that the program must impart to reach the final behavior desired 
of the learner. The subject-matter is prepared in the form most 
applicable to the training task. Practice and drill materials are 
incorporated in the body of the program as required. Where ap- 
propriate, instruction and practice in the use of equipment and 
other job-connected materials are included. 

3. Program Editing and Testing. As sections of the program 
are completed during the content-development process, they 
are submitted to an editorial and testing process. Content ac- 
curacy is carefully checked. The logic and order of the program 
is evaluated to insure that each instructional item provides the 
necessary information. Each item is carefully appraised to see 
that it is based on preceding items and that it prepares the learner 
for items to follow. 

The testing of the program is accomplished empirically. As 
each section of the program becomes available, it is given to a 
small sample of the ultimate learner population. The results of 
this test are carefully evaluated, and content material is modi- 
fied and restructured as necessary. 

On completion of all sections of the program, the final copy is 
submitted to a final editing and testing process. The complete 
program is again tested with a representative sample of the user 
population to validate its utility and effectiveness in terms of the 
specified training objectives. 
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Question: Hoiv does programed instruction compare with con- 
ventional instruction ? 

Answer: The classroom lecture is the principle tool tradition- 
ally employed for teaching. Although it affords a highly efficient 
and economical means of communication between teacher and 
learner, the lecture has several inherent drawbacks. The rela- 
tionship between lecturer and learner is basically passive, and 
most of the learner's responses are covert. The lecture mate- 
rials are necessarily designed for group use although each 
learner can respond only in accordance with his individual intel- 
ligence, motivation, and attention. The provision of stimulus 
situations that set the stage for appropriate responses from the 
learner is entirely dependent on the skill and intuition of the 
lecturer who, for the most part, has no means of recognizing 
or reinforcing the learner's covert behavior. 

An attempt is made to correct some of the deficiencies of the 
lecture by periodic tests that check the adequacy of the learner's 
behavior. However, even the most efficient tests make no pro- 
vision for the correction of errors at the most critical time in 
the learning process when such errors first occur. Thus, al- 
though the testing technique can readily detect errors in learn- 
ing, the correction of such errors must be achieved subsequently 
by appropriate re-education. This makes for an inefficient and 
uneconomical system of education. 

Programed instruction overcomes the basic shortcomings of 
the lecture method. It insures control over the teaching rela- 
tionship by demanding successful interaction between the learner 
and the material to be learned. It makes appropriate use of re- 
wards to reinforce corrective responses and, thereby, insures 
that effective learning occurs. Programed instruction provides 
the required educational functions of teaching, testing, and correc- 
tive instruction (where necessary) automatically and simultane- 
ously. It permits the learner to proceed in terms of his individ- 
ual capability and time. It frees the instructor to use his time 
and skills in a more direct relationship with the learner. 

Question: What are the important terms in the field of pro- 
gramed instruction? 

Answer: The important terms in the field of programed instruc- 
tion are the following: 

a. Framea single instructional item, including informational 
material, question, and answer. Frames are, generally speaking, 
the constituent elements of a program. 

b. Linear program a program in which the sequence of frames 
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is fixed and independent of the learner's responses. The order of 
presentation of the frames is the same for each learner and does 
not vary whether or not the correct responses are made. Exam- 
ples of linear programs are given in Appendices A, E, F, H and I. 

c. Branching program-a program in which ancillary informa- 
tional paths are provided in addition to the main informational 
sequence. Ancillary paths are selected for presentation in ac- 
cordance with the learner's response pattern. They are utilized 
to provide a vehicle for review and for presentation of additional 
information. Appendix G illustrates a typical Crowder-style 
branching program. 

d. Cue or prompt aids that are incorporated in some frames 
to enable the learner to respond correctly. Several different 
forms of cue or prompt are illustrated by the following frames 
(taken from a program in Business Mathematics prepared by 
H. Huffman and S. Margulies): 



10. Most employees contribute part of 
their salary for insurance. The 
Federal Government provides in- 
surance for an employee in the 

Fed Ins Contributions 

Act. (Complete the words) 

11. Federal Insurance is financed 
partly by employees. The Con- 
gressional Act establishing this 

insurance is called the F , .. 

I C Act. (Complete the 

words) 

12. "Social Security tax" is a common 
non-technical term for F.I.C.A. tax. 
What act establishes this tax? 



10, Federal Insurance 



11. Federal Insurance 
Contributions 



12. Federal Insurance 
Contributions Act 



Of the three frames illustrated, frame 10 provides the most ex- 
treme example of prompting and 12 the least. 

e. Fade or vanish-terms applied to the gradual decrement of 
cues during a program. As cues and prompts are faded out, re- 
sponse difficulty is increased. The reduction in cueing strength 
through the three frames illustrated above demonstrates the ap- 
plication of vanishing. 



SECTION 2. Internal Industrial 
Programs 



This section is devoted to a consideration of the techniques by 
which three companies developed programs for their own use. 
These early projects were aimed at answering the fundamental 
questions that every company must consider: Is programed in- 
struction truly feasible in industry? Does it result in decreased 
training time or increased efficiency? Do employees accept it? 
Do machines play an integral role? 

In the first article, the early experiences of Eastman Kodak 
are summarized. The dearth of commercially available indus- 
trial material, the decision to develop programs internally, the 
selection of program writers, and the initial employee reactions 
are outlined. Particular attention is given to the general require- 
ments for an efficient machine and the question of whether or not 
to use a microfilm storage device. Other important problems, 
such as administrative procedures, preparation time, and the 
selection of subject-matter areas, are also discussed. 

The second article, based on an IBM report, is directed at an- 
swering the question: Is programed instruction really worth- 
while? The material conventionally taught in 15 classroom 
hours was organized into a programed textbook, and compari- 
sons were obtained between class room -trained and program- 
trained students. Trainees who learned this material through 
programed textbooks took less time, obtained higher final ex- 
amination scores, showed greater uniformity of performance, 
and enjoyed the course more than students exposed to conven- 
tional material. 

The importance of machine presentation of programed instruc- 
tional material has been a widely debated issue. The third arti- 
cle is based on a Bell Laboratory study that is important be- 
cause it was done under industrial conditions using a lengthy 
program and a machine especially designed for this material. 
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Results indicate that machines and textbooks work equally well. 
This report also provides data indicating how many hours a day 
students can use programs and, also, the average instructional 
time required for bright and dull students. 



Programed Learning and Teaching Machines 
in Industrial Training 



By JEROME P. LYS AUGHT 
Eastman Kodak Company 



Wherever teachers, educators, and industrial trainers have 
turned for the past several months, they have been confronted 
with the evidence and challenges of a new force in learning. Call 
it automated teaching or programed learning or what you will 
the data demand our attention. From the pages of Fortune, Time, 
Business Week, the newspapers, the journals, including the jour- 
nal of the AS 723, headlines and commentary cry out for an awak- 
ening on our part to the happenings in educational experiments. 
Already claims are mounting that here in our own time we are 
witnessing the greatest breakthrough in education since the 
printing of the first textbook. Because these claims are made 
seriously by men of proven ability they dare not be ignored. 

On the other hand, because most of the reports to date have 
concerned the experience of teachers and students in schools 
and universities, with little emphasis on experimental develop- 
ments within business and industry, it may be possible that the 
applications to the adult level have been minimized or even neg- 
lected. For this reason, we feel you may be interested in the 
experiences of the Eastman Kodak Company in applying these 
new techniques to their ongoing program of industrial education. 

In order to present our information in some form of order, we 
would like to devote this article to our experiences with develop- 
ing learning programs and programers. We also would like to 
trace for you our investigation of the equipment phase of auto- 
mated teachingthe teaching machine. In a strong sense, this 
makes for proper emphasis as well as convenience. The pri- 
mary question, of course, deals with the educational worth of 
the program through its use does the student actually learn, 
effectively, efficiently? The secondary question involves how 
we wish to present such a program to the individual learner. 

The Eastman Kodak Company employs about 47,000 people in 
the United States and 74,000 world wide. This will give you some 
understanding of the industrial training potential that we must 
face. Trends develop in every human activity, and, if there has 
been any single recognizable trend in training within our com- 
pany, it has certainly been toward a more and more individual- 
ized approach, based on specific, personal needs. There have 
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been a number of reasons for this increased emphasis on indi- 
vidualizing our training efforts, and our evaluations attested to 
its effectiveness. 

Approximately two and a half years ago, we first became ac- 
quainted with the theoretical field of programed learning. Be- 
cause it seemed to offer a means for sharply improving our in- 
dustrial training methods, and because it seemed uniquely de- 
signed to emphasize the individual aspects of application and 
rate of learning, we were intrigued. Within a few weeks, we had 
embarked on the research and development of programed learn- 
ing for industrial applications. 

PROGRAMED LEARNING IS NOT NEW 

We began, of course, by broadening our acquaintanceship with 
the published materials in the field. We entered into corres- 
pondence with those who were working at the scholastic and 
collegiate levels. We talked time and again with all those whom 
we could reach that were actively engaged in programed 
learning utilization. To our mingled relief and chagrin, one of 
our earliest discoveries was that programed learning was not 
the new-born babe we had initially been led to expect. Rather 
it was the culmination of effort that began at least 35 years 
ago with Sidney Pressey at Ohio State, and had, in more recent 
times, been revitalized by the efforts of B. F. Skinner and his 
associates at the Harvard Psychological Laboratories. From 
1955 on, we learned programed learning had been taken under 
study at a score of colleges and research centers and even to 
the classrooms of elementary and secondary schools. While 
there was hardly anything new under the sun, we wondered how 
we had contrived to stay so long in the shade. 

PROGRAMED-LEARNING THEORY IS UNIQUE 

Programed learning, we found, was not to be lumped with or 
confused with the large number of learning techniques and de- 
vices which are regularly advanced in the journals. Audio- 
visual aids, for instance, contribute much to the learning pro- 
cess, but they are inherently stimulus devices, and are not a 
complete learning system. Programed learning is unique in 
theory and in practice for it offers the student a complete learn- 
ing " cycle." Because this was a key understanding we had to 
get across to our industrial training people, it might be best to 
describe briefly what is involved in student use of programed 
learning, and relate it to the widely-accepted stimulus-response 
learning cycle, so that we can then stand on common ground. 

The learning program consists of a carefully constructed 
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pattern of learning bits. After each presentation of information, 
the student is immediately given an opportunity to answer a 
question, construct a solution, solve a problem, or in other 
ways to demonstrate that he has acquired mastery of this bit 
of learning. As quickly as he responds, he is given the oppor- 
tunity to compare his answer with the program answer to de- 
termine his own effectiveness. All of this takes place between 
the individual and the program, and is paced by the student 
himself. 

Two alternatives now follow. If the student has made the cor- 
rect response to the stimulus item, confirmation of his correct- 
ness by the program serves as immediate reinforcement. And 
any classroom teacher knows the satisfaction and reward of a 
student's learning and simultaneously knowing that he is learn- 
ing. On the other hand, if the student has responded incorrectly, 
he obtains the advantage of immediate knowledge of his error. 
He need not wait for a later class period or examination paper 
to recognize a mistake. In psychological terms, we apply im- 
mediate extinction to incorrect performance. 

This describes what happens as a student responds to a single 
stimulus item. The learning program for a single term may 
consist of some 2,500 to 5,000 items. These items are so inter- 
connected and designed that the student will constantly progress 
to higher degrees of complexity. At the same time the program 
introduces review and recall items enabling him to retain and 
master the course material. 

In all this, the learning program is designed to be self-con- 
tained and self-sufficient so that the student can learn the 
material through his active interaction with the stimulus items. 
Many educators feel that with properly programed stimuli the 
student will not only master the material but do so with almost 
complete freedom from error since he should always be pre- 
pared by previous items to answer correctly the item next 
placed before him. 

THERE ARE FEW INDUSTRIAL PROGRAMS 

While our initial efforts had unearthed a goodly volume of 
theoretical knowledge, we found that the actual number of devel- 
oped learning programs was few. And most of those which were 
available were essentially non-industrial subjects. That is, 
while those we met spoke highly of the work Douglas Porter was 
doing in 6th grade spelling, this did not seem to be a program 
which we could well anticipate for use within the Eastman Kodak 
Company. Some of the existent programs did offer more direct 
application, particularly those designed for college level courses. 
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It was obvious, however, that if we were to get any real dis- 
tance run out of programed learning, at least in the foreseeable 
future, we would certainly have to program many of our own in- 
dustrial subjects. Other companies looking into the field came 
to the same conclusion, and this reinforced our desire to make 
an effort to program. 

Our initial venture with hardy nods to our compatriots from 
IBM was to program the interpretation of punchings in standard 
data cards. There are a number of cards which people handle at 
Kodak which have various information punched on them usually 
on the line where your signature is supposed to go. There being 
a great deal of natural curiosity about the meaning of these 
holes, we were assured of inquisitive souls and volunteer stu- 
dents. Additionally such material would have practical applica- 
tion to our training efforts for electronic data programers. 

Initial Kodak Efforts 

So we began to program. In the main, we had to break our own 
way for there was very little published material on the techniques 
of program item construction. We evolved our own principles, and 
rewrote them as we experienced preliminary student feedback. 
At the end of several weeks and the development of 70 frames 
of a programed course-to our delight, and some amazement, 
people could actually read the punchings in the data cards. 

Having determined that programing for industrial training was 
possible, we set out to determine its practicality. Many of our 
areas of training were explored in order to set up some reason- 
able priorities for programing effort. Necessarily, each of these 
was to fulfill a bonafide training need since we intended to study 
cost, effectiveness, and student reaction to the new approach. 

It was also evident that we needed to train additional pro- 
gramers. This we began to do. The trainees were chosen to re- 
present not only industrial training people, but also production 
and staff departments. The training program consisted of an 
extended workshop with emphasis on the construction of an act- 
ual programed sequence. (Incidentally, we now have one man 
programing how to program.) Not all of our trainees developed 
into programers, but approximately 75% of our "graduates" 
have subsequently programed industrial training course material. 

Programing Experience 

Some of the findings which must at this point still be quite 
preliminary indicate that there is some correlation between 
intelligence and programing skill, and between knowledge of 
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subject matter and the ability to program that matter. More im- 
portant than either of these factors, however, is the deep, per- 
sonal interest of the individual in devoting himself to this work. 
I am suggesting that a programer must be intelligent, knowledg- 
able, and beyond this, dedicated to producing a new medium 
which will idealize the learning situation for the student. Fortu- 
nately, these people do appear, and they are constantly buoyed 
up by seeing the results of their efforts reflected in student 
performance. 

We also discovered that the rate of individual programing var- 
ied considerably our results are too fragmentary to suggest 
why with any great accuracy. On the overall, by the time we add 
in the time required for evaluation and refinement of the items, 
and redesign of items to keep us within a 5% error ratio, we 
find we spend between 30 and 60 minutes per frame of a finished 
program. 

Because programing represents an investment in time and 
personnel, we have directed our own efforts at producing pro- 
gramed materials in those training areas where we cannot ex- 
pect an early introduction of commercial programs to the mar- 
ket. Where we know programs are being developed commer- 
cially, we are content to wait and evaluate their efforts. In the 
meantime, we are going forward, emphasizing financial justifi- 
cation, and training mastery. 

Evaluation of Early Results 

We began some critical evaluative studies of learning results. 
This effort is being made by a joint team of industrial trainers, 
industrial psychologists, and programed learning personnel. The 
reports of the studies will be made available upon their conclu- 
sion since we anticipate they will be of importance in judging the 
importance of programed learning on industrial and all adult- 
learning. 

The comments of a few of our students might be of interest. 
They reflect the general tenor of remarks, and have not been 
chosen to show a particular bias: "I had to give it (the program) 
my complete attention/' "It forces intense concentration." "Felt 
you were made to stay right on the problem." "It was intriguing. ? ' 
"I think I learned quite a bit although it was all verbal."' "I 
learned something. Iwas familiar with the subject, but it clarified 
terms." "Completely dumb on the subject; learned quite a lot." 

These are the experimental beginnings, yet they point the way 
to greater effort. Our programers and our students are con- 
vinced that here we have in our hands a new and forceful tool to 
improve learning. We have already initiated programing in these 
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areas: logarithms, slide rule, basic photography, theory of sen- 
sitometry, industrial relations standard procedures, supervisory 
training, materials handling, and economics. We intend to con- 
tinue and expand these efforts. 

Up to this point I have hardly made mention of the teaching 
machine, the presentation device of programed learning. Natur- 
ally, when we had begun our efforts at programing., we came up 
against the question of how to present such materials to the stu- 
dent. To answer this, we undertook a concurrent research and 
development program in teaching machines while we worked on 
programs and programers. 

As we began to train additional programers, and to select ini- 
tial subject-matter areas, we had to consider the manner in 
which we would present programed materials within the company. 

Sometime before this, we had received from Dr. James Hol- 
land a copy of the psychology course which he and Dr. B. F. 
Skinner had prepared at Harvard.* This course came in the form 
of sheets of paper intended for use on their disc-like machine. 
The items of the program were printed in such fashion that, 
as the disc was rotated on the machine, each item would appear 
in sequence under windows on the cover of the device. In order 
to show illustrations, figures, or pertinent data relating to pro- 
gram items, a separate sheet of paper or "panel" was utilized. 

Simple comparison of items from the Holland -Skinner pro- 
gram and our own indicated that we would have to pursue an in- 
dependent study of presentation. Figure 2-1-1 illustrates the 
problem. Our items were of varying length, but many of them 
were longer than any items in the psychology program. More- 
over, as we had gained experience in programing, we found it 
increasingly desirable to include within the frame any graphic 
representations that were necessary rather than employ sep- 
arate sheets or panels. 

Experiments with Devices 

For the sake of experiment, we placed our initial programs 
on 35mm slides which were then projected by the student using 
conventional Kodak Signet and Cavalcade projectors. The items 
were projected onto a small screen, and the automatic advance 
mechanisms of the projectors enabled the student to progress 
through the material with a minimum of effort. This easily 
solved the problem, of graphic illustrations, but it left much to 
be desired. One problem, for instance, was that two slides were 



^Recently published as a programed textbook "The Analysis of Human 
Behavior. M by McGraw-Hill, New York. 



PROGRAMED LEARNING AND TEACHING MACHINES 



29 



required for each item in the program one to present the stim- 
ulus; another the response. This struck us as being unnecessar- 
ily cumbersome and expensive. 

A second method we tried used a Kodak Signet projector and a 
filmstrip of the program. This was less expensive than the sep- 
arate slides, but still entailed two projections per item unless 
we wished to modify the projector. 
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Fig. 2-1-1 A comparison of items constructed for use in the Holland- 
Skinner psychology course and in our industrial sensitometry course. 
Note the difference in overall size, and also our emphasis on placing 
graphic material within the frame of the item to which it refers. 

A third experimental set-up was devised which actually 
masked the lower portion of a 35mm slide, and, by means of a 
solenoid, the mask could be moved, and the response exposed 
for comparison. This solved the problem of two projections and 
suggested that, for certain applications, it might be possible and 
desirable to present learning sequences by means of standard 
audio-visual equipment with perhaps only minor modification. 

Programed Books 

As we made a series of studies on conventional film devices, 
we also considered the possibility of simply using programed 
textbooks or workbooks. At least in the early stages of program 
construction and revision this format seemed to be the simplest 
and least expensive means of presenting the program. The items 
could be typed or handwritten on cards or sheets; a sufficient 
number of copies could easily be obtained by the Verifax pro- 
cess; and we could quickly have the program ready for trial and 
evaluation. Our programers soon suggested a number of for- 
matsincluding one gleefully dubbed "Kodatext" in order to 
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Fig. 2-1-2 Various programed textbooks, workbooks, and worksheets 
that have been utilized within the Industrial Training Departments of 
Eastman Kodak Company. 

do necessary testing of the programs. Figure 2-1-2 illustrates 
some of these programed workbooks. 

We liked the sheer simplicity of the programed "book/' but 
there were some sobering second thoughts. The volume of paper 
consumed was considerable. Though we were just getting under- 
way, it was already obvious that paper costs and handling would 
be significant. As the Klaus and Lumsdaine article elsewhere 
in this volume points out, their physics program for 500 students 
"required over 3 1/2 tons of paper and 120 dozen loose-leaf note- 
books." It was one thing to deal with the workbook at the experi- 
mental level, it would be something else to adopt this as the 
medium for industrial training materials. 

Programing Variations 

On top of these considerations, it was evident that not all pro- 
grams would follow the Skinner modeL At least three variations 
were in use or under development. Norman Crowder had suc- 
cessfully programed materials using what he termed "intrinsic** 
programing. This differed from the Skinner material in that it 
characteristically employed multiple -choice responses, and re- 
quired that the student have random access to the program, 
based on his choice of answers, rather than that the progress be 
in strict sequential order. Robert Glaser, among others, had 
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spoken of the desirability of having a provision for " skipping' ? 
items in an otherwise sequential program, thus permitting the 
rapid learner to by-pass items which might seem redundant to 
him, but which were desirable for those students encountering 
difficulty with the program. John Barlow spoke of still a third 
form of programing which had come out of the work done by the 
Earlham College research project. Known as "conversational 
chaining/' it was a variant of the Skinner model. 

These diverse forms of programing made it difficult to adapt 
standard audio-visual equipment to the task of performing as a 
teaching machine. At this point, we felt that our whole effort to 
develop industrial programs was in jeopardy because of the 
" device' ? situation. Certain teaching machines began to appear 
on the market, and we studied each one in search of an answer 
to our needs. In some cases, they might represent a short-term 
solution, but with our concern over the eventual costs of paper 
and paper-handling, and our desire to limit our machine costs, 
we turned once more to the possibility of designing equipment to 
meet precisely the specifications recommended by our experience. 

Machine Requirements 

While we had not formalized any "design objectives, 73 we 
knew that our minimum requirements for an individual teaching 
machine would include: 

a. Versatility The ability to present the various types of pro- 
graming that might be necessary. 

b. Standardization While we wanted to present variant forms 
of programing, we wanted to do so by means of a standard re- 
production format. 

c. Economy We were extremely conscious of the cost factor, 
and were determined to find a simple, inexpensive means of 
presentation. 

At this point, we held a series of meetings with the engineer- 
ing groups located in our Apparatus and Optical Division. We 
stressed to them our belief that the programs and the program - 
ers must dictate the necessary equipment, but that it was vital 
to the success of programed learning that we find devices 
which would offer versatility, standardization and economy. 

Some form of film device seemed the answer to our needs. 
In this belief, we were certainly not unique. Klaus and Lums- 
daine had concluded in their study, "it is likely that the pre- 
sentation of auto -instructional material on film may be the most 
economical procedure." Many other individuals had expressed 
the belief that, over the long run, a film device could best sat- 
isfy the needs of the field. 
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After the initial explanatory meetings with the engineers, and 
after considering various applications of filmstrips and 16mm 
motion picture cassettes, we narrowed our study to an explora- 
tion of microfilm handling equipment. Its known capability for 
miniaturization, and its economy in reproduction meant that we 
would be starting in a promising direction. 

Planning and development now began in earnest with the design 
engineering staff of Recordak, the subsidiary of Eastman Kodak 
which specializes in microfilming. Having again explained the 
needs of the program, and how we wished to utilize this equip- 
ment in industrial training, we reviewed the devices that might 
be adapted. There were several standard devices that could han- 
dle microfilm programs, and could allow the sequential and ran- 
dom access that was required. Unfortunately, these devices were 
designed to handle rolls and magazines of microfilm and by a 
little calculation it appeared that a single roll of 16mm micro- 
film could contain the entire Holland- Skinner program, and at 
least another of similar length. Our reaction was that this would 
mean having the entire course "in use" whenever a single stu- 
dent was working on a single lesson. 

At this point a parallel development came to our rescue. Quite 
independently of our own research, a Recordak engineer had been 
developing a new concept of microfilm unitization, looking to 
greater flexibility in the handling of "book" records-on-micro- 
film than is afforded by the conventional 100 ft roll form. His 
slide-track card innovation permits single or multiple strips of 
micro-film to be sandwiched in card mounts. Although the film 
strips are not visible, the individual images can be projected one 
by one on the screen by manual movement of the film strip within 
its own slide-track (see Fig. 2-1-3). 

Programed material could be microfilmed, cut into strips on 
the basis of lessons or units, inserted in the card, and used on 
the teaching machine as required. Additional lessons or units 
could be stored or filed until needed by the student. No single 
student need ever "monopolize" more than four or five units 
at a time, and, of course, far fewer "sets" of any given pro- 
gram would be required to handle large groups of students. The 
card concept offered an additional advantage. It was the means 
for filing, storage, and handling built right into the teaching 
machine system. 

In order to utilize the card system, a new microfilm reading 
device had to be designed and constructed, but this presented 
a relatively minor problem once the means of handling the pro- 
gramed materials was determined. The reader would have an 
indexing device which would permit either sequential operation, 
with the facility for skipping certain items, or a non-sequential, 
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Fig. 2-1-3 Slide-track card used for storing, filing, handling, and pro- 
jection of programed materials in the Recordak Mentor Teaching Ma- 
chine. One unit of film is partially withdrawn to demonstrate how it is 
utilized within the device. 

random-access operation. The viewing screen would be large 
enough to permit the incorporation of illustrations, graphs, 
charts, and other materials into the program. 

Machine Prototypes 

In June 1960 we had our first prototype (Fig. 2-1-4). Known 
as the Mentor, it employed strips of 16mm film. It permitted 
the handling of both sequential and intrinsic programs, and it 
had an auxiliary answer device which could be employed for 
use with constructed response items (Fig. 2-1-5). Throughout 
the summer of 1960, we tested and studied the equipment, and 
experimented with the filming of programs. In the fall of 1960, 
we put the device on display at the seminar sponsored by the 
Foundation for Research on Human Behavior. Since that time, 
we have started building additional models of this first prototype, 
with only slight modifications as indicated by our experience, 
for use in Eastman Kodak. 

While the first model of the equipment had been designed to 
meet the needs of variant forms of programing a point we feel 
will be more important as time passes we recognized that for 
the present and immediate future, most programs under con- 
struction are patterned after the Skinner model. For this reason, 
we constructed a second model of the device which was consider- 
ably smaller and less expensive to build (Fig. 2-1-6). This sec- 
ond prototype, or Mentor Model 2, can present standard sequen- 
tial programs, and can "skip" appropriate items as required; 
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Fig. 2-1-4 The Recordak Mentor Model 1 Teaching Machine in use in 
industrial training. In this illustration, the device is being used in the 
unlocked mode for non-sequential subject matter, and the optional an- 
swer device is not in use since a constructed answer is not required. 




Fig. 2-1-5 The Recordak Mentor Model 1 Teaching Machine shown in 
use on an industrial subject. Because this is a sequential, constructed 
answer-type program, the answer device is employed in this mode of 
operation. 



PROGRAMED LEARNING AND TEACHING MACHINES 35 

It utilizes the same basic film and envelope system so that com- 
patibility can be achieved. 

Equipment Design Factors 

The factors which we feel are most important in the design of 
the equipment, the factors which are vital to our concept of teach- 
ing machine utilization, are these: 

a. Machine capability for handling various program formats 
and programing techniques even within the same lesson or unit. 

b. Provision for integrated use of graphic illustrations directly 
within the program frame, 

c. Simplicity of producting microfilm programs from typed 
or printed original materials. 

d. Compactness and convenience of program storage through 
miniaturization. 

e. Durability of programs including re-usability and freedom 
from student or machine damage. 

f. Capability of using color materials, at low cost, when desir- 
able or required by the program content. 

g. Ability to include "cheat-proof" guards against student 
violation of the program design use of the guards being at the 
option of the programer, 

h. Capability of producing either short-run or long-run quan- 
tities of programs at reduced material cost. 

i. Utilization of programed materials by lesson or course unit 
to permit reduced program overhead. 

j. Machine operation of an extremely simple and relatively 
service-free device meaning uncomplicated student use and little 
repair. 

Our plans for future development of such devices include the 
addition of audio facilities to the "system" so that the student 
may hear and record appropriate stimuli and responses. More- 
over, the systems approach to the teaching machine lends itself 
to the possible addition of various recording devices which may 
be desirable from the standpoint of program research and evalua- 
tionapart from the normal use of the machine in the learning 
situation. 

All of the research and development done on our equipment 
was directed toward its use by the Eastman Kodak Company for 
industrial training within our own plants. Following the construc- 
tion of the first prototype, consideration naturally broadened to 
include possible commercial applications of the device. In the 
fall of 1960, the Educational Microfilm Systems Department was 
established in the Recordak Corporation. We were convinced by 
this time that microfilm is the most versatile and satisfactory 
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Fig. 2-1-6 The Recordak Mentor Model 2 Teaching Machine that is 
used for sequential, constructed answer type programs with the facil- 
ity for " skipping " items as directed by the program. 

medium for preparing and utilizing efficient systems for pro- 
gramed learning. 

The new department was formed to offer technical assistance 
in systems and microfilm techniques to individuals and organiza- 
tions engaged in the assembly and use of educational material in 
programed format and to continue the study of design features 
for microfilm equipment that will be required, Eastman Kodak 
Company has continued the research and development of learning 
programs and learning sequences using standard audio -visual 
equipment, so that, depending on the nature of the students, the 
material and the learning task, an effective means of program 
presentation may be selected. 

In the process of investigating this field, we have raised some 
questions, the answers to which have had a strong influence on 
our thinking. Because others have been asking similar questions, 
our tentative conclusions may be of interest. In time, we expect 
to have firm answers. 
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WHY WORRY ABOUT LEARNING PROGRAMS AT ALL? 

Industrial training will always require new approaches and 
new conditions. Two trends in training needs, however, seem to 
call specifically for the examination of programed learning. The 
first trend is that toward higher and more complex human skills. 
The mechanical engineer must keep abreast of changing develop- 
ments in electronics and hydraulics because our new integrated 
systems require inter -disciplinary handling. The second trend 
is related to the first. We must further individualize our train- 
ing efforts because the job needs of our people will become more 
unique, and we will find a rising proportion of training assign- 
ments involving a single individual, or at most a very limited 
number. 

Programed learning seems promising in light of both these 
trends. Programs can teach higher skills, and areas like mod- 
ern mathematics, electronics, thermodynamics, and the physical 
sciences are already being programed. Just as importantly, the 
learning program is designed to be auto-instructional. The stu- 
dent will learn individually by mastering the program. 

BUT IS PROGRAMED LEARNING EFFECTIVE IN PRACTICE? 

In a photography course, the students had to work with loga- 
rithms in order to master sensitometry. In any class one could 
expect a number of individuals who needed to learn about loga- 
rithms, or at least have a brief refresher course. This meant 
other individuals had to mark time; it also meant the time of an 
instructor. One programer produced a sequence of 103 frames 
on logarithms. By mastering this unit, the student learned all 
he needed to know to solve problems in sensitometry. Since that 
time, it has not been necessary to conduct classes in logarithms, 
or hold up class progress, or employ an instructor to teach in 
this connection. The students schedule themselves, perform the 
programed sequence on a teaching machine, and master loga- 
rithms quickly. 

In another case, the clerical staff of a department needed skill 
in statistics. A pre-test showed most scores ranked in the for- 
ties and fifties. The girls took the first three units of a pro- 
gramed course in descriptive statistics. They scheduled them- 
selves over a six-week period to meet the convenience of their 
office workload. At the completion of the material, the lowest 
score on the post-test was in the nineties. Supervisory personnel 
were so impressed that they began the programed course. 

One more example: another programer was teaching a unit on 
human motivation by means of lecture -discuss ion. He programed 
the same information; then conducted an experiment. Two groups 
of supervisors were given the learning program only; three 
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groups of supervisors attended the lecture-discussions. Later 
tests showed that the " program' ' students retained twice as 
much of the motivational spectrum developed by the instructor. 
Thirteen specific questions showed the " program" students ex- 
celling on nine,, most by significant differences. One question 
showed no difference. Three showed the " conventional 77 students 
had a higher percentage of correct answers. The instructor con- 
cluded that the students who had worked individually with the 
material had gained equalif not highermastery without bene- 
fit of class or instructor. This being his first attempt at pro- 
gramming, it is safe to assume there will be even greater im- 
provement in the future. 

WHERE DO YOU OBTAIN THE PROGRAMS? 

For the time being, this is a problem. Programed materials 
are just now becoming commercially available, and it will cer- 
tainly be some time before the training specialist will have a 
choice of programs in a wide variety of subjects. A rapid growth 
in the number of available programs is anticipated, however, be- 
cause the publishers are obviously accelerating their efforts to 
reach the market. The problem may well be to limit publication 
to effective programs. 

Nevertheless, we expect to continue programing sequences in 
areas unique to our own training needs. Currently we are work- 
ing on some 24 projects related to our instructional courses. 
Few, if any, of these would have general applications outside the 
Eastman Kodak Company , and most companies will face the same 
situation. General fields, such as statistics, will be covered by 
commercial programs. Unique company needs must be self- 
programed, or developed by consultants under contract, 

HOW DO YOU DETERMINE THE SUBJECTS 
TO BE PROGRAMED? 

If a suitable program is available for purchase, we use it. If 
it is material which we must program for ourselves, we select 
it on the basis of criteria which include: 

a. Known difficulty with the present training approach (as in 
the case of the unit on logarithms discussed above). 

b. Number of students to be trained, 

c. Permanency of the subject-matter. 

d. Estimated time for development. 

If we know a present training method leaves much to be de- 
sired, and if we have a fairly large number of students who will 
need this material over a period of time then it would seem to 
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be a suitable area to program. The subject matter should be 
relatively stable, otherwise the program will quickly become 
obsolescent. Development time is an obvious consideration in 
assigning priorities. 

We have learned one valuable lesson in selecting a subject for 
programing. Rather than program a "course," we break out 
units or lessons and program these. This means that the mate- 
rial can be fitted into the next presentation of the course. Thus, 
we have prompt field-testing and feedback of results. Program - 
ers like this approach because it gives them known dates for 
using and evaluating the sequence before proceeding on to ad- 
ditional units. 

The units can then be fashioned together into a sub-course or 
course. After programing the unit on logarithms, a unit on sen- 
sitomety was developed; now other areas of photography are 
being programed. Eventually they will probably be combined 
into a total course in basic photography. We will be gaining 
benefit from each unit of the course, however, as soon as it 
is completed. 

ARE SOME SUBJECTS MORE DIFFICULT TO PROGRAM? 

Generally, the more logical the subject-matter, the more eas- 
ily it can be programed. This is why there are several programs 
in algebra, and very few in the social sciences. 

There is no reason, however, to believe that more difficult, 
conceptual learning cannot be programed. Work is being done on 
several languages, the Old Testament, philosophy, and various 
sciences. Our own 24 areas of programing include units from 
every portion of our industrial training curriculum. We wanted 
to find out if there was some subject which could not be pro- 
gramed. We have found none to date. 

Programing in some areas has proceeded at a much faster 
rate than in others. Our tentative conclusion is that the differ- 
ence is as much a function of the programer as it is of the ma- 
terial. 

HOW DO YOU OBTAIN PROGRAMERS? 

Initially, two of us learned to program by trial and error. 
Later, we wanted to train a number of programers so we de- 
veloped a two -week workshop covering principles and techniques 
of programing. The students sampled programs developed by 
others, and then constructed short practice units of their own. 
This approach was refined by experience, and some twenty peo- 
ple have now been trained within the company. 

Not all the students developed into programers. A number of 
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them did not enjoy the work. On the other hand, at least two indi- 
viduals became interested in programing as a result of reading 
articles in the local press. They talked with us, took away litera- 
ture and programs, and tried their hand at the game. Both be- 
came excellent programers. 

Recently, our prople participated in a course in programing 
held at the University of Rochester. The students were teachers, 
administrators, and industrial training people. That course, plus 
our experience, and three summer workshops planned for this 
year will give us an increased opportunity to determine the qual- 
ifications of people who make good programers. Now it is known 
that a programer must be intelligent, resourceful, somewhat 
creative, and knowledgeable in the subject matter. Given these 
qualifications, he can be taught the techniques and the methodology. 

WHAT ARE THE COSTS AND TIME INVOLVED 
IN PROGRAMED LEARNING? 

The commercial programs that we have used have ranged in 
price from $4 to $20. We feel that this cost, and the investment 
of time by the person taking the course, have been most reason- 
able when measured against learning effectiveness. 

The cost of developing our own programs must, of course, be 
taken into account. At present, our programers spend 20-30 
minutes per item. This includes initial writing, sequencing, and 
first revision following field testing. Additional time must be 
spent on later revisions. This rate of programing permits sub- 
stantial savings in later instructional time, and student learning 
time. 

Most of our programing has been done in areas that are now 
being taught by conventional methods. If we were to program 
entirely new material, it would, of course, take more time than 
if we were to develop lesson plans and lecture notes. Programed 
sequences are much more demanding and require more rigorous 
analysis of the subject matter. 

HOW DO YOU ADMINISTER PROGRAMED LEARNING? 

We provide private rooms equipped with teaching machines 
and programed textbooks. The student may reserve the room at 
any time during the day that fits his own schedule. He then pro- 
ceeds through the program at his own rate. 

Machines and programs have also been made available to in- 
dividuals to take to their desks. The special room, however, 
tends to reduce interruptions, and is a superior learning environ- 
ment. In almost all cases, this training takes place within 
the regular working hours of the individual. 
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WHAT DO THE PEOPLE THINK ABOUT 
PROGRAMED LEARNING? 

Student reaction has generally been quite favorable. Many 
found it exciting, " I thought it was wonderful! ", "It was very 
intriguing! " . More of them found it demanding, challenging. 
"It forces intense concentration." "I had to give it my com- 
plete attention." Almost all had the feeling they were learn- 
ing. "I learned a lot." "I wanted to go on." 

To learn more about how people felt, and particularly what 
applications they had made of programed learning, we conducted 
a survey of the supervisors who participated in the experiment 
on teaching human motivation. The results are shown in Table 
2-1-1. 

Table 2-1-1. Results of Survey of Supervisors 

Percent answering "Yes" 



Lecture- Programed 
Question discussion learning 



Have you given any thought 
to this material? 


73 


100 


Have you discussed it with 
anyone? 


60 


78 


Have you. tried to explain 
it to anyone? 
Have you made any effort 
to apply some of its ideas ? 


34 
41 


57 
100 



This leads to the tentative, but very interesting, conclusion 
that programed learning students come away from their experi- 
ence with a higher disposition toward action and application. 
This is all the more important if you define industrial training 
in terms of "changing behavior." 

Those supervisors whose people have had training by means 
of learning programs have, in almost all cases, been impressed 
with its efficiency and effectiveness, and there have been in- 
stances where they have taken the material themselves as a 
result of student comment. 

Then there are the training men. Not all of them have been 
enthusiasts. As evidence mounts that people do learn by this 
approach, more instructors have been attempting to write pro- 
grams, and asking to be kept up-to-date on available commer- 
cial programs related to their field of activity. 

There have also been negative reactions -primarily from the 
students. Some simply "do not like" programed learning. They 
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say they are accustomed to classes, teachers, and regular text- 
books, and do not wish to use such a different approach to learn- 
ing. Several who felt this way acknowledged, however, that they 
had indeed learned material by means of the program. A very 
few of the students seem particularly ill-at-ease with the ma- 
chine. They felt better about using the programed textbooks. A 
more serious problem, though, is the contradictory feeling of 
students that the same material is developed too rapidly or too 
slowly for most effective learning. This difficulty must be stud- 
ied much more thoroughly, and can probably only be solved by 
constructing parallel, but differently paced, sequences. 

IS THE TEACHING MACHINE IMPORTANT? 

We definitely think so. We know students can learn by simply 
using the learning program. We believe, however, that the ma- 
chine enhances learning, and contributes positively to the learn- 
ing process. 

The programer designes his material so that students will 
encounter each new bit of information in proper order. This can 
be better guaranteed by means of a mechanical device than by 
handing the student a programed textbook which he may or may 
not use as intended. The device allows recording of student's 
progress through the material, and provides data to analyze, in 
order to improve subsequent student use of the material. 

Those of you who have encountered some of the variant forms 
of programed textbooks, with scrambled pages, reversed read- 
ing, and up -side down printing will sympathize with our desire 
to eliminate impediments between the student and his mastery 
of the program. We believe the teaching machine gives the stu- 
dent a convenient, simple, and straight -forward means of reach- 
ing his goal, and that this is most desirable. 

Cost may seem an odd reason to cite for using teaching ma- 
chines. It appeared, at first, that a simple system using paper 
sheets would be the cheapest method we could find. The sheer 
volume of paper consumed, and its preparation, filing, storage, 
and handling far exceeded the initial cost estimates. We began 
a research study on teaching machines largely because we 
wanted economy and miniaturization. We found, however, that 
microfilm permitted us to do the job of training, and effect sav- 
ings in the process. We are now employing the Recordak men- 
tor teaching machines in our industrial training programs. 

A final thought on the use of teaching machines is that they 
permit handling all known variations in programing because 
their presentation to the student can be standardized through 
the common " format' ' of the microfilm. This eliminates the 
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need for struggling with various shapes and kinds of media and 
materials . 

Our students like the ease and clarity of machine presentation. 
They tend to find the material more "absorbing" and "demand- 
ing" when used in the machine. There are a few who simply do 
not like using teaching machines, but most negative comments 
concerned mechanical and design deficiencies in the earliest 
experimental models. 

THE FUTURE OUTLOOK FOR PROGRAMED 
LEARNING IN INDUSTRIAL TRAINING 

You can see that we are optimistic. It will be 3-5 yr before 
a wide variety of programed materials is generally available. 
If interest continues at the present rate, however, that time 
may be greatly reduced. 

Without doubt, publishers will make more rapid strides in 
offering programs to the market; universities and research 
centers will produce more definitive information on learning 
behavior, programs, and programers; and more companies 
will undertake the development of programed materials to fit 
unique training needs. 

Our experience of the last three years is proof to us that 
time spent in exploring industrial applications of programed 
learning is well spent. We have found such learning to be effec- 
tive, efficient, and economical. It has given us new insight into 
adult learning, and has provided us with new opportunities to 
improve the learning experience. 



The Effectiveness of Programed Instruction: 
Experimental Findings 

By J. L. HUGHES 
International Business Machines Corporation 



A series of three experiments was conducted in October, No- 
vember, and December of 1960 that compared the effectiveness of 
programed instruction with conventional classroom instruction. 
The first 15 hr of the IBM 7070 training course for customer 
engineers given at Poughkeepsie were selected for this compari- 
son. The programed-instruction material (see Appendix A) was 
presented by means of programed textbooks. Six experimental 
classes totalling 70 men and two control classes containing 42 
men were used. The results were as follows: 

a. Programed instruction reduced the time needed for present- 
ing the subject matter from 15 hrtollhr, a saving in time of 27%. 

b. On a comprehensive test of the material covered, the pro- 
gramed-instruction group averaged 95.1, compared with 86.2 for 
the control group. The difference of 8.9 points represented a 
learning gain of 10%. The programed-instruction group also 
showed a smaller dispersion of scores, with 89% scoring 90 or 
above compared with only 43% for the control group, 

c. Replies to a questionnaire administered to the programed- 
instruction group were highly favorable. Of the group, 87% liked 
programed instruction better than conventional classroom instruc- 
tion, and 83% favored using programed-instruction in future 
courses in place of conventional instruction. 

Thus programed instruction is immediately applicable to re- 
ducing the number of days a student spends at a central training 
center and, eventually, to decentralizing the teaching of many 
courses to branch offices with resulting substantial economies. 
In addition, the improvement in learning possible with this tech- 
nique should produce better trained sales, systems, and service 
staffs. Customers could also be trained more efficiently. Pro- 
gramed instruction could be expected, therefore, to aid sales and 
customer -relation activities. Management, technical, manufac- 
turing, and clerical training also could be expected to benefit 
from the use of programed instruction. 

A DESCRIPTION OF THE EXPERIMENTS 

In March, 1960, a team composed of a customer-engineering 
instructor and a psychologist was formed to prepare programed 
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textbooks for the Introduction and Programing Section of the 16- 
week course on the IBM 7070 given to customer engineers at 
Poughkeepsie. The services of a psychological consultant from 
the American Institute for Research were also secured for one 
day a month to review the materials prepared by this team. 

By the end of September, 719 frames of programed-instruction 
material covering the first 15 hr of the IBM 7070 course were 
completed. The topics covered were the names and functions of 
units of the 7070; data flow; bit coding; instruction, data, and in- 
dex words; and the computer-program step. To test the effective- 
ness of these materials, the following experiments were carried 
out: 

a. Two classes of 21 students each that began during Septem- 
ber were designated the control classes. They were taught the 
material by two different instructors in 15 hr using the conven- 
tional IBM classroom method (lecture-discussion). To measure 
what they had learned, these classes were administered a com- 
prehensive 2-hr examination covering the entire instructional 
period. A sample of the questions contained in this examination 
is given in Appendix B. 

b. Six classes consisting of 70 students made up the total ex- 
perimental group. Two of these classes reported for training 
each month from October through December, 1960. They were 
instructed by means of programed instruction materials only. 
The instructors remained in the classrooms but confined their 
activities to passing out the programed instruction materials 
and briefly answering the few questions put to them by the stu- 
dents. These classes also took the comprehensive subject-matter 
examination. In addition, they completed a questionnaire asking 
them to evaluate programed instruction (Appendix C). 

To test the comparability of the control and experimental 
groups on various background data, information, such as age, 
educational level, length of customer -engineering experience, 
and length of previous computer experience, was collected by 
means of another questionnaire (Appendix D). A test of mental 
ability (Programer Aptitude Test) also was administered. 

THE RESULTS 

The subject matter covered in these experiments took 15 hr 
when presented by the standard classroom method. The same in- 
formation was covered in 11 hr by programed textbooks, a saving 
of 27% in presentation time. 

On the comprehensive 2-hr test covering the material pre- 
sented in the entire instruction period, the control group had a 
mean of 86.2 compared with 95.1 for the experimental group. 



46 INTERNAL INDUSTRIAL PROGRAMS 

The experimental group thus scored an average of 8.9 points 
higher, a learning gain of 10%. This difference was found to be 
statistically significant at the 0.01 level. 

Not shown in Appendix B are the mean test scores for each of 
the three successive experimental groups, consisting of two 
classes each, that started training in October, Novermber, and 
December, 1960. These means were 95.7, 95.0, and 94.7, re- 
spectively, which show the consistency of learning achievement 
when programed instruction is used over a period of time with 
different classes. 

The distribution of test scores for the experimental and control 
groups is given in Table 2-2-1. Note the greatly reduced disper- 
sion of achievement for the experimental group. Almost all (89%) 

Table 2-2-1. Means, Standard Deviations, and Distributions of 
Achievement-Test Scores for Control and Experimental Groups 



Achievement 
test 
score 


Control 
group 
(=42) 


Experimental 
group 
( = 70) 


N 


% 


N 


% 


95-100 


5 


12 


49 


70 


90-95 


13 


31 


13 


19 


85-89 


7 


17 


5 


7 


80-84 


9 


21 


3 


4 


75-79 


6 


14 






74 and below 


2 


5 






Totals 


42 


100 


70 


100 


Mean 


86.2 




95 


.1 


Standard deviation 


7.4 




4. 


.0 



of the experimental group scored 90 or above on the examination, 
with the rest scoring in the 80 ? s. For the control group, however, 
only 43% scored 90 or over, and the rest ranged down into the 
70's. (The standard deviation of the experimental group was 4,0 
compared with 7.4 for the control group.) One of the effects of 
programed instruction thus appeared to be that more trainees at- 
tained a high level of achievement in the course. 

There were no significant differences between the experimen- 
tal and control groups in age, educational level, months of cus- 
tomer-engineering experience, or previous computer experience 
that affected tne achievement test scores. The experimental group> 
however, was significantly higher (58.2 vs. 51.2) on the Pro- 
gramer Aptitude Test (PAT), a measure of mental ability (see 
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Table 2-2-2. Means and Standard Deviations on Background 

and Aptitude -Test Variables for Control and Experimental Groups 

and Their Correlation with Achievement -Test Scores 



Control group Experimental 
Variable (w = 42) group (n = 7Q) 


Product -moment 
correlation 
with achievement- 
test score 


Age (years) 






Mean 


28.8 29.3 


-0.01 


S.D. 


5.7 5.4 




Attending 






college (%) 


17 34 1 


0.22 2 


CE experience 






(months) 






Mean 


62.3 63.1 


0.01 


S.D. 


57.2 38. 8 3 




Previous com- 






puter experi- 






ence (%) 


38 43 


0.31 2 


Programmer 






Aptitude Test 






Mean 


51.2 58. 2 4 


0.44 2 


S.D. 


12.1 8.5 4 





difference in educational level (percent attending college) between 
the two groups appeared to be large, but it was not statistically signifi- 
cant. Educational level, however, correlated significantly (0.22) with 
achievement -test scores. Since educational level also correlated signi- 
ficantly (0.31) with PAT scores (which were used to adjust achievement- 
test scores), no additional adjustment of achievement-test scores based 
on educational level was considered necessary. 

Correlation coefficients of 0.25 and 0.22 or higher were found to be 
significant at the 0,01 and 0.02 levels, respectively. 

3 The difference in standard deviations for the experimental and control 
groups on length of customer-engineering experience was significant at 
0.02. In view of the low correlation (-0.01) of customer-engineering expe- 
rience with achievement test scores, however, no adjustment in achieve- 
ment test scores was made for this variable. 

Differences in PAT means and standard deviations for experimental 
and control groups were significant at 0.02; none of the other differences 
between means and proportions was significant. 

Table 2-2-2). It was also found that PAT was significantly corre- 
lated (0,44) with the trainees' achievement-test scores. A statis- 
tical technique (analysis of covariance) was used, therefore, to 
adjust the achievement test scores to eliminate the effect of dif- 
ferences in PAT scores between the two groups. 

When the achievement test scores were so adjusted, the experi- 
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mental and control group means became 94.7 and 86.7 respec- 
tively. The difference between the means of 8.0 points was only 
slightly less than the difference of 8.9 for the unadjusted means 
and remained significant at the 0.01 level. It was concluded, 
therefore, that the difference in achievement between the experi- 
mental and control groups could not be attributed to the effect of 
differences in aptitude test scores. 

On the student Questionnaire administered anonymously to the 
six experimental classes, the replies of the trainees were very 
favorable to programed instruction (Appendix C). Of the total 
group of 70 men, 87% liked programed instruction more than con- 
ventional instruction, and 83% said they would prefer using it in 
future IBM courses. Only 6% liked programed instruction less 
than conventional instruction, and only, 13% would have some ob- 
jections to using it again. 

Most of the dislike and objections occurred in one class where 
the instructor, in recording the time for completing different 
units of the materials for the purpose of the experiment, caused 
the students to feel that they, and not the materials, were being 
timed. In other classes, where the timing was done less obviously, 
the dislike and objection to programed instruction was much less. 

It was interesting to note that practically all of the students 
realized the advantages of programed instruction over conven- 
tional instruction. All of the group stated before taking the ex- 
amination that programed instruction was more effective than 
conventional instruction, and 93% also found it less difficult. In 
addition, 60% reported that programed instruction required less 
home study. This suggested that the learning gain might have 
been achieved with greater savings in total class and home study 
time for most of the students than indicated in this experiment, 
which measured only class time. It also should be noted that it 
was the opinion of observers that the class time could be further 
reduced without significantly affecting learning achievements. 

The Student Questionnaire also provided the trainees with space 
to indicate what they particularly liked (Item 6) or disliked (Item 
7) about programed instruction. They were also given the oppor- 
tunity to write in any additional comments (Item 8) that they 
wished to make. Appendix C summarizes the most frequent com- 
ments made regarding each of these items. 

The responses to Item 6 (liked) gave the main reasons for the 
trainees' favorable reaction to programed instruction. These 
included relative speed and ease of learning, individual nature of 
instruction given, and logical presentation of material. The com- 
ments on Item 7 (disliked) related mostly to complaints about the 
timing, mechanics, repetition, difficulty of review, and boredom 
from too much PI without classroom discussion. This type of 
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comment should be greatly reduced in the future with improve- 
ments in materials and better integration of these materials with 
class discussion groups. The responses to Item 8 were similar 
to those included under Item 6 and 7, and also contained a number 
of recommendations for improving the materials and using them 
in future IBM courses. 

A DISCUSSION OF RESULTS 

The promising results obtained from this series of experiments 
indicate the potential advantage in time, cost, and achievement by 
applying programed instruction to other areas of training. These 
results corroborated the positive findings found by other investi- 
gators in training experiments outside of IBM. 

These findings also suggested several applications to training 
programs that promise important economies. The immediate one 
is the reduction in the number of days that students need to spend 
at training centers in a given course. This reduction can be trans- 
lated immediately into savings in the direct, daily-living expenses 
of these students and, eventually, into reductions of other educa- 
tional costs. 

The second ultimate application is the decentralization of 
training by enabling people to be trained in basic courses at the 
branch offices or customer's offices. Because trainees work in- 
dividually on programed instruction materials, a training package 
can be prepared for distribution to the field that will teach the re- 
quired subject matter. The possible economies in this method of 
instruction are easily visualized. 

Another result of programed instruction would appear to be 
better trained sales, systems and service groups, with attendant 
advantages to sales and customer relations that cannot be esti- 
mated directly. Similarly, customer training also could be ex- 
pected to improve, thereby enabling customers to use the equip- 
ment more efficiently and, again, facilitating good customer rela- 
tions. In management, technical, manufacturing, and clerical 
training areas, programed instruction could be applied to similar 
advantage. 

In spite of the additional commitments in funds and manpower 
that setting up program -writing teams would require at the pres- 
ent time, any company can ill afford to postpone the long-term 
gains and economies in training programs that programed instruc- 
tion promises. But there is the need for more research on the 
programed instruction technique. Because programed instruction 
is still in its infancy, a number of improvements in technique 
leading to greater efficiency still can be expected. 



Books as Teaching Machines: Some Data 

By HOWARD O. HOLT and JOSEPH HAMMOCK 
Bell Telephone Laboratories, Incorporated 



Since the early days of the current wave of enthusiasm for 
teaching machines or self-instruction methods, there has been a 
prevailing question as to whether programed materials presented 
in book form could teach as well as comparable materials pre- 
sented by machines. Informal reports have indicated that many 
persons who have worked with both media have given up machines 
and have turned to using books entirely. At the time this study 
was initiated, no published evidence bearing on the relative effi- 
cacy of books and machines was available. Since then, however, 
there have been two studies published as institutional reports 
one by Eigen and Komoski (1960) and the other by Roe, et al. 
(I960) both of which have supported the position that books are 
as effective as simple machines. 

Because we felt that an adequately controlled comparison of 
books and machines would be of value, the present study was in- 
troduced into an already planned, larger study of programing and 
auto-instructional methods vs. traditional methods. The gist of 
the finding is that, under the conditions of the study, the notion 
that programed books teach equally as well as programed teach- 
ing machines cannot be rejected. The remainder of this article 
describes the conditions under which this result was obtained 
and presents our view of its implications for the choice between 
books and machines. 

The present study seems to have the following unique aspects 
when compared with the studies cited: 

a. It is based on a much longer course; it took subjects an 
average of 51 hr to complete the course. 

b. Learning in the course is concentrated so that any differen- 
tial fatigue, boredom, or motivational effects can accumulate, 
(Specifically, the subjects worked 6, 7, or 8 hr a day, depending 
on school policy, for five days a week.) 

c. Subjects (technicians) are drawn from a lower aptitude level 
than the college and high-school students in the other studies. 

d. The study provided for replications across three cities in 
widely different locations. 

The course content is basic electricity, which normally is 
taught as the fundamental course in a series concerned with elec- 
tronics maintenance in the telephone business. The materials 
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were programed for auto -instruction by a team consisting of Dr. 
Myron Woolman, a psychologist, and Mr. Glenn Valentine, an 
electronics supervising instructor from a telephone company. 
The program, in general, is of the Skinnerian type that requires 
constructed responses, but it deviates in the following ways: 

a. Some of the items are purely informational and do not re- 
quire a response. 

b. Periodic diagnostic tests are given, after which the learner 
either goes ahead or repeats portions of the program. 

c. The learner has an answer list of some 30 items in front 
of him at all times. 

The student makes a total of about 3,500 constructed responses 
as he works through the program. 

The teaching machine used was one designed and constructed 
by the Bell Telephone Laboratories because, primarily, at the 
time the study started, a suitable machine could not be found on 
the market. Figure 2-3-1 shows the machine at a student work 




Fig. 2-3-1 

position. The machine has the usual provisions for the frame or 
item to appear in a window. The student writes his answer in 
context, and then turns a knob that results in his answer moving 
up under glass and in the confirming answer being exposed. In 
operation, this machine is much like the Rheem -Calif one Didak 
and the Foringer Machine. Figure 2-3-2 shows the machine open 
for loading. The program is printed on 5-in.-wide paper tape in 
rolls of up to 225 ft in length. 
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Fig. 2-3-2 

Figure 2-3-3 shows the books or booklets that were used in the 
study. The program in the book is identical to that presented by 
the machine. The book materials are fastened into two rings of 
a four -ring notebook. The student reads an item and writes his 
response in context. He then turns the page and sees the confirm- 
ing response on the next page. The next frame is on the follow- 
ing page and so on. The subjects were instructed not to look 
ahead or to turn back but to use the book as if it were a machine. 
However, the policing of this procedure was not at all rigorous, 
and "cheating" was observed to occur. 
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Fig. 2-3-3 



Two predictor or control variables were measured prior to 
the commencement of training for use in covariance adjustments. 
These were the following: 

a. An IQ estimate based on the Otis Employment Test (Test 2, 
Form B), 

b, The degree of the student's prior knowledge of electricity 
fpre -course performance on items from course tests). 

The program was administered to two successive classes in each 
of three Bell Telephone companies. Each class consisted of sub- 
jects who were assigned at random to the book and machine treat- 
ment conditions. Table 2-3-1 shows the experimental design and 
number of subjects completing the course. Originally, there were 

Table 2-3-1. Experimental Design and Size of Sample 



Cities and 
replications 



Book 



Machine 







' 



II 
Total's 



4 
4 

4 
3 

6 

7 

28 



5 
5 

5 

5 

6 
9 

35 
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Otis IQ (Transformed to mean = 50) 

Fig. 2-3-4 Time on program as function of Otis IQ for subjects 
with IQ's less than mean. 

34 and 36 subjects assigned to books and machines respectively. 
Of these, 28 and 35 completed the course. This shows a loss of 
six subjects on books and one on machines. Of the six losses on 
books, two were due to illness and three were probably due to 
low IQ's. Figure 2-3-4 shows the scatter plot for IQ and time to 
complete the course for all subjects with IQ's below the mean, 
including the five subjects who did not complete the course be- 
cause of slowness. 

Just by chance, the four subjects with transformed* IQ's of 
less than 30 were assigned to the book group. Because the re- 
gression of time to completion on IQ indicates that the mean time 
to completion for the group with IQ's less than 30 would be greater 
than the 77 hr allowed for completion, it is assumed that the fail- 
ure to finish for the three out of four subjects in this group can 
be attributed to low IQ's rather than to their using books. Also 
shown in Fig. 2-3-4 are the two subjects who did not finish but 
whose transformed IQ's (37 and 42) would predict that they should 



* Transformed IQ's were obtained by subtracting a constant from stand- 
ard Otis IQ's. 
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have finished. It should be noted that one of this pair worked on 
books and one on a machine. 

Table 2-3-2 shows the mean comparisons between books and 
machines for the two predictors and three criterion variables. 
It can be seen that aptitude was slightly lower in the book group 
but prior knowledge of the course material was somewhat higher. 
As for the criterion variables, average times to completion were 
identical, and the small differences on the two criterion tests 
were in favor of the book group. 

Table 2-3-2. Mean Comparison of Books Vs. Machines 
Variables Books Machines 



Predictors 






Aptitude (Otis)* 


49.3 


50.8 


Knowledge 


21.1 


17.6 


Criteria 






Time 


51.0 


51.0 


Facts 


54.0 


52.3 


Concepts 


57.7 


56.9 



*Standardized with Mean of 50 

The magnitudes of the correlations between the predictors and 
the criterion variables ranged between 0.47 and 0.68. Toward 
the end of providing every opportunity for the null hypothesis to 
be rejected, a multiple covariance factorial analysis was per- 
formed for each of the criterion measures and adjustments were 
made for both of the predictors. The analysis was done with fac- 
torial partitioning of variance for cities and classes within cities 
so as to remove these sources of variance from the error term. 
The resulting comparison of adjusted means is shown in Table 
2-3-3. None of the values of F for the book-vs. -machine compari- 
sons was larger than 1.0, so the null hypothesis remains tenable 

Table 2-3-3. Group Means for Criterion Measures Adjusted for 
Multiple Covariance in Otis Scores and Prior Knowledge 

Criteria Books Machines 

Time 50.96 51.01 

Facts 53.38 52.78 

Concepts 57.22 57.28 

for each of the criterion variables. Books and machines can be 
viewed as equally efficient with respect to costs in student time 
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and output in factual knowledge and/or conceptual facility under 
the conditions of this evaluation. 

The principal implication of the study is that, for operational 
conditions approximating those of the present evaluation, vari- 
ables other than student time and instructional effectiveness 
should be used in deciding between machines and books for pro- 
gramed instruction. Such variables would be cost, preference, 
administrative covenience, etc. 

* * * 

This article is based on a paper presented to a meeting of the 
Midwestern Psychological Association at Chicago, May 6, 1961. 
The authors wish to acknowledge the valuable assistance of 
George S. Rotter of Bell Telephone Laboratories, who performed 
the data analysis. 
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SECTIONS. Military Programs 



United States military organizations have contributed a great 
deal to the field of programed instruction. Much of the basic 
research for this field has been done by the military or under 
military contract. Also, many of the existing programs, machines 
and simulators have been developed under military auspices. 
The work of the military in the field of programed instruction 
is of particular interest to those concerned with industrial train- 
ing problems and academic education because the totality of mili- 
tary education encompasses both of these areas. This chapter is 
also of particular interest to those investigating the military as 
a possible market outlet for machines, programs, and services. 

The first article of this section is an overview of the needs, 
research, development, and other activity of the military in the 
programed instruction area. Of particular interest is the break- 
down of the different organizations within the military and their 
interest and/or contributions to the field of programed instruc- 
tion. This article presents information on the nature of the acti- 
vities in different military organizations and on where further in- 
formation can be obtained on these activities. Virtually all types 
of training activities, from verbal learning to manual skills, are 
included in the organizations treated in this article. 

The short article by Col. Hosmer and Capt. Nolan presents 
data on the training time saved by the use of programed instruc- 
tion for courses in air-police training, medical training, and per- 
sonnel training. Those interested in the feasibility, time, and 
monitoring economy of training will be interested in this summary 
of concrete practical applications of programed instruction for 
military training purposes. 

The next article in this section presents an example of one type 
of system used to teach Russian to potential prisoner-of-war in- 
terrogators. This article has been selected for inclusion in this 
section because it is a typical military training problem with an 
exemplary system designed to solve the problem. 
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"Teaching machines in the modern military organization " is 
a theoretical paper that discusses the areas of the potential use 
of various types of devices. Descriptions of these devices are 
given, and some of the support and implementation problems are 
discussed. 

The four articles that constitute this section are by no means 
the totality of work done in the field of programed instruction 
in the military. In the opinion of the editors, however, these 
articles are representative of the work done by the military, and 
the reader is urged to consult the bibliography in the article by 
Bishop and Ragin for more information. 



Programed Instruction in the Armed Forces - 

an Overview 



By C. K. BISHOP and J. J. REGAN 
U. S. Naval Training Device Center 



Training of and by the armed forces has always been sensitive 
to educational innovation a good deal more sensitive than the 
educational community at large. This is due to a variety of fac- 
tors. First, the objectives of military training are relatively well 
defined, since the behaviors or skills to be developed have been 
rather specifically identified. The range of skills required is 
more circumscribed than are those of secondary and higher edu- 
cation in the civilian community. Armed forces training has been 
principally concerned with perceptual-motor and procedural tasks. 
Additionally, since military training is more directive and less 
encumbered by formal diploma requirements, the reaction time 
for change is reduced. Anew idea or teaching device can be tried 
out systematically while still in the embryo stage. Finally, the 
need for shortening training time and improving quality has in- 
creased continually since World War II. The pool of qualified 
manpower is shrinking, the length of service is short, and the 
complexity of military systems is increasing at an amazing rate. 

Since the climate for innovation is favorable and the need for 
maximum efficiency in training crucial, the military services 
support a substantial effort in educational research both in their 
own laboratories and through contracts to universities and indus- 
try. The Army, Navy and Air Force all support important pro- 
grams of educational research. 

This research and development has led in pilot training, for 
example, to the successful use of flight simulators a decade be- 
fore their introduction into nonmilitary training. The armed forces 
have pioneered in the development and exploitation of audio visual 
aids, and their integration into the curriculum is today more ad- 
vanced than in public education. The overhead projector, for ex- 
ample, now widely available commercially, was developed by the 
U.S. Naval Training Device Center. Again, in the area of educa- 
tional television, the Navy was conducting controlled experimenta- 
tion in its validity for training and had an operating, experimental 
TV facility in 1951. Comparisons were made among TV instruc- 
tion, films, and "live" instruction (Rock, Duva, and Murray, 1951). 
It should not be surprising to learn then that, in the "dark ages" 
of programed learning that existed from the time of Pressey's 
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first promulgation to the Skinnarian renaissance, various individ- 
uals working with and within the military were experimenting 
with this type of teaching (Fry, Bryan, and Rigney, 1959). 

The purpose of this chapter is to describe those features of 
military training which make programed learning particularly 
useful and important, and to cite illustrative examples of military 
contributions and current research in automated teaching. 

MILITARY TRAINING 

Something was said in the introduction to this article of the fea- 
tures of armed forces training which give rise to and permit a 
special interest in educational innovation. Much of this applies to 
programed instruction -an innovation of widespread interest 
and importance to the entire educational community. There are, 
however, some special conditions that obtain in military training 
that make programed instruction of particular interest. These 
conditions can be described under the headings of personnel and 
subject matter. 

Personnel 

Personnel problems are twofold instructors and students. 
These are admittedly universal problems, but they assume a pro- 
portion in military training which makes programed instruction 
especially desirable and useful. 

Instructors in the military tend to be subject matter specialists 
rather than teachers as such. They are drawn from various tech- 
nical fields and characteristically have little teacher training. 
Their competence is restricted to specific technical areas where 
they are, indeed, experts. This fact, combined with the character 
of the subject matter and the extremely limited time available, re- 
quires a highly organized and structured curriculum. Instruction 
proceeds according to detailed hour-by-hour study plans. 

Instructor teaching loads are heavy even by present day public 
educational standards. Instructors are rotated in and out of teach- 
ing assignments so that no stable pool can be maintained. Release 
from military service, promotions and enticements from industry 
all act to augment the problem. 

Since one of the effects of programed instruction is to stand- 
ardize not only course content but, more important, to standardize 
the way in which such content is presented, it is obvious that such 
a technique would prove particularly useful in military training. 

Subject Matter 

Course content in the military is typically highly technical. 
With the increased emphasis on mechanization in all branches, 
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technical "know-how" is a prerequisite. Not only must theory be 
taught and learned, but also considerable skill in operating, main- 
taining, and troubleshooting equipment must be acquired. 

Whether it is sound pedagogy or not there is a dichotomy be- 
tween the "nice-to-know" and the "must-know" courses. The 
shortness of time available for classroom instruction, the needs 
of the field for skilled technicians, and the rapid turnover in per- 
sonnel compel the military to teach the minimum amount of ma- 
terial that must be learned. Furthermore, the program must be 
accelerated so that the enlisted student is cramming material 
eight hours a day, five days a week and often for eight to ten weeks 
at a time. In addition to the heavy classroom load, the student 
must also do extensive "homework" assignments each evening. 
It is not unusual for such courses to continue for 18 to 20 weeks 
on this type of schedule. 

With such a concentrated time schedule there is little or no 
opportunity for remedial work or special help for the student. 
Even if the student is learning at a satisfactory rate there will be 
problem areas which are difficult for him to master. For example, 
in the field of basic electricity the concepts of resistance, reac- 
tance, and capacitance may be particularly troublesome. Well- 
constructed programs of self -automated tutoring can provide the 
student with the necessary information to help him overcome dif- 
ficulties in these areas. 

Because of the lack of instructors and as a result of the poor 
preparation many inductees have received in secondary education, 
a backlog of students frequently accumulates before there are 
classes available to admit them. During those weeks while the 
students are awaiting class assignment, self-automated tutors 
with programs pertinent to the school could be given as refresher 
courses. Considering the great heterogeneity of the incoming 
classes, it is especially important that at the start of a course the 
enrollee be brought up to a minimally acceptable level for efficient 
learning to occur. Such programs, of course, require no instruc- 
tor and they efficiently utilize the student's time. 

A kindred problem facing the military is the maintenance of 
skilled performance on those complex systems that are never 
activated except in times of national emergency. Some of these 
systems are so critical that little or no practice is possible and 
the only available training is at training centers which use expen- 
sive and complex training simulators. Maintaining highly complex 
skills over extended periods of time with no opportunity for prac- 
tice is difficult. It is possible that appropriate teaching machine 
programs for maintaining such skills can be devised. Although 
the intricate problem of transfer of training is involved here, it 
should be pointed out that those skilled performances which con- 
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sist of motor acts, i.e., muscular activity, are rarely purely 
"motor." There are verbal components in any acquisition of a 
muscular activity. Under the conditions mentioned here there is 
no reason why practice on the verbal aspect cannot be effected. 
Also, there is ample evidence that pictorial representations of 
stimulus objects can lead to high positive transfer (Gagne, 1958). 
In other words, under certain conditions there is little loss of 
transfer if one learns from pictures rather than from the real ob- 
ject. This concept of stimulus similarity and also the verbal com- 
ponents of a skill could be presented with teaching machines, and 
since the aim is to maintain associations that have already been 
learned, the transfer should be high. Automated tutors of the 
Crowder type could well be used in these situations. Not only 
could procedures training be maintained, but appropriate pictorial 
representations of relevant malfunction displays could be shown 
and the student interrogated as to the proper course of action to 
be taken. 

Continuing with the same line of thought, learning and main- 
taining troubleshooting skills for those systems can be done with 
machines. Frequently, the troubleshooting process is largely an 
intellectual activity, namely, that of problem solving and examin- 
ing the cause-to-effect relationship. Conventionally, troubleshoot- 
ing and data flow are taught through a student and a skilled instruc - 
tor on actual components of a particular system. The rub, once 
again, is the availability of instructors and equipment. An auto- 
mated tutor, however, can be made to simulate the equipment and 
the instructor insofar as the information input to the student is 
concerned. 

Even the nonverbal aspects of perceptual -motor behavior can 
be developed through programmed instruction. "Adaptive" ma- 
chines such as those envisioned by Pask (1960) can program in- 
puts (stimuli) to the student as a function of his behavior. As the 
students acquire skill the values of the dimensions of the system 
can change, approaching the "real life" situation. 

The subject matter of military training, then, since it tends to 
be concrete and factual and since it must be presented in highly 
concentrated form, is particularly suited to programing. The 
need for and feasibility of automated instruction are clear. The 
balance of this chapter will be devoted to a description of repre- 
sentative research and application efforts by the military services. 

STUDIES IN PROGRAMED INSTRUCTION 

The development and application of programed techniques are 
proceeding in all three services with particular attention to those 
areas where special problems exist. Continuing interchange of 
information and materials takes place among the services although 
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actual studies are conducted independently. Each branch, of course, 
while involved in various aspects of automated teaching, has its 
own emphasis. 

Army 

Electronics training is one of the major problem areas in all 
the services. The requirement for skilled electronics personnel 
is increasing daily both in quantity and quality. One of the Army 
Technical Services (the Signal Corps) conducts a substantial 
amount of electronics training for the Army. The School Text Di- 
vision in conjunction with the Education Research Division at Fort 
Monmouth, New Jersey, is programing 32 hours of the fundamen- 
tals of electricity. This will be a linear program. Plans are being 
made to compare this program with conventional methods. A 
similar effort is also underway at the Southeastern Signal Corps 
School at Fort Gordon, Augusta, Georgia. This latter program, 
also linear, is concerned with communication procedures. 

The Army Rocket and Guided Missile Agency has been sponsor- 
ing programed instruction research at the Electronics Research 
Laboratory of Columbia University. They have compared pro- 
gramed instruction with reading manuals and reading statements 
in the area of missile fire control orientation (Robinson, 1961). 

Several studies are being conducted by the Human Resources 
Research Office (HumRRO) of George Washington University. A 
course in Russian specifically directed to military needs has been 
programed. This group is also working with the Signal Corps in 
programing maintenance material. Finally, the Human Factors 
Division, Chief Office of R&D, U. S. Army, is participating in 
studying the integration of programed learning with closed-cir- 
cuit television. 

Navy 

The Navy has placed primary emphasis to date on investigating 
experimentally certain variables associated with programed 
instruction. A number of programs have been prepared in elec- 
tricity and mathematics principally for the purpose of investigating 
these variables. In addition, at least two field applications of pro- 
gramed learning are currently undergoing test. 

Office of Naval Research. The Office of Naval Research is 
sponsoring several studies designed to contribute to the theory 
and application of programed learning. This agency is support- 
ing, on a continuing basis, basic research at the University of 
Southern California and at the American Institute for Research. 

Although ONR's interest is primarily theoretical, a study is 
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being made by the University of Southern California in trouble- 
shooting and refresher training. At Treasure Island, California, 
the learning of troubleshooting by either a scrambled book or by 
a linear program is being evaluated as a substitute for laboratory 
experience. These two techniques are also being evaluated for re- 
fresher training by shore -based personnel prior to their return 
to the fleet. 

Naval Training Device Center. Another naval agency besides 
ONR that has been actively engaged in research is the Naval 
Training Device Center. From 1956 it has been investigating 
such variables as the nature of context cues (Silverman, 1960a), 
randomness versus sequential presentation (Zuckerman, 1961), 
response mode, pacing, and motivational effects (Silverman, 1961), 
and a preliminary field validation study at a naval school. 

At the time of this writing a field validation study in the use of 
programed learning is being conducted at the Service School Com- 
mand, Great Lakes Training Center. The contractor is Teach- 
ing Machines, Inc., supported by the U. S. Naval Training Device 
Center. Programs have been developed in DC and AC electricity 
for the Navy's Electricity and Electronics Preparatory School. 
In addition, a course in the calculus for electronics has been con- 
structed for an advanced electronics school. Since many enlisted 
men have poor preparation in the basic operations of arithmetic, 
a refresher mathematics course has also been provided. 

A cursory description of the experimentation to be conducted 
in the Preparatory School is as follows: The population from 
which the experimental and control groups will be randomly drawn 
consists of enlisted men chosen to be electronic technicians. Half 
of the experimental group will learn electricity with a teaching 
machine and the other half with a programed book. Control 
groups will consist of classes taught by the traditional lecture 
method and by closed-circuit television. All groups will be equated 
on GCT-ARI scores and the criterion tests will consist of the 
school's regular examination as well as an electricity test inde- 
pendent of the naval school. 

Further implementation of this study will be conducted by the 
Personnel Measurement Research Branch of the Bureau of Naval 
Personnel who will replicate the first experiment with larger 
samples, investigate retention effects, and determine the relation- 
ship between individual differences and programed learning. 

Other Naval Installations. A partial list of naval agencies either 
actively engaged in or having shown interest in programed learn- 
ing are the following: 

a. The Chief of Naval Air Technical Training at Memphis, Tenn- 
essee, who is currently developing programs in electrical courses 
relevant to their teaching needs. 
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b. The Bureau of Supplies and Accounts, Washington, D. C., 
which is contracting for programs in storekeeping and retail 
sales. 

c. The San Diego Naval Base where experimental work is being 
conducted on the feasibility and utilization of branching programs 
in the teaching of electronics. 

d. The Submarine School at New London, Connecticut which has 
put on board Polaris submarines mathematical programs developed 
by Harvard University. These programs, carrying college credit, 
are used in conjunction with teaching machines, 

e. The Guided Missile School at Dam Neck, Virginia, which has 
assigned scrambled books on computer arithmetic in the learning 
of basic computer courses. 

Coast Guard 

The Coast Guard Academy at New London, Connecticut, is de- 
veloping programs and using teaching machines. 

Marine Corps 

The Marine Corps at Quantico, Virginia, has evinced interest 
in the programing of foreign languages for refresher training of 
their officers who will be stationed at oversea posts. Currently, 
they are investigating the feasibility and utilization of various 
teaching machines and how they may be implemented by the Corps. 

Air Force 

The Behavioral Sciences Laboratory of the Wright Air Develop- 
ment Division is intensively conducting research. Some aspects 
of their program of research are the development of automated 
instructional programs and facilities, effective displays of infor- 
mation in training systems, effective ordering of information for 
automated training, assimilation of information during automated 
training, and motivational aspects (see the article by Eckstrand, 
Rockway, Kopstein and Morgan in this section). Historically, they 
have shown a marked interest in this form of teaching and several 
of the well-known names today in the field of programed learning 
developed their early interest through the research done in the 
Air Force. 

WADD is sponsoring two programs at the University of Pitts- 
burgh. One program is in the area of general science for junior 
high school students. Two different formats have been constructed, 
one a linear format and the other a spiral format. The second pro- 
gram on the university level deals with the mathematical bases 
for management decision making. This, also, is in two different 
formats, one on a single track and the other a multi-track version. 
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With the Educational Testing Service, WADD is also investi- 
gating the relationship of mental test theory and automated pro- 
grams. 

Keesler Air Force Base Project. The Keesler Air Force Base 
experiment in basic electronics is being continued after the com- 
pletion of the preliminary experimentation (Benson and Kopstein, 
1961). The automated material is intrinsically programed and 
ultimately there will be a three-way comparison of the effective- 
ness of the instruction using a machine, a scrambled book, and 
the conventionally taught course. 

Other Air Force Installations. Agencies that are either actively 
investigating program learning or have shown a marked interest 
in the use of this type of instruction are the following: 

a. The Air Force Academy at Denver, which is in the process 
of training instructors in the art of programing courses to be 
taught at that institution. 

b. The Strategic Air Command, which is using a variety of pro- 
grams for individual study at various sites. 

c. The Medical Service School, Gunter Air Force Base, Ala- 
bama, which has evinced interest in the use of automated instruc- 
tion for the training of medical technicians. 

d. The Air Force Systems Command at Bedford, Massachusetts, 
which has been sponsoring the American Institute for Research for 
the investigation of self -instructional programs for SAGE system 
operators. The primary aim is to develop and evaluate automated 
tutoring techniques and materials as they pertain to the training 

of basic job knowledges and skills for command and control opera- 
tor tasks within the SAGE Track Monitor and Intercept Director 
positions (Shettel, et al.. 1961). 

STATE-OF-THE-ART INFORMATION 

In addition to the current efforts at programing and field try- 
out described above, the armed forces have produced under con- 
tract a number of critiques and summaries of the field of pro- 
gramed instruction. They include the following: 

a. Two well-known studies sponsored by the Office of Naval Re- 
search: an extensive bibliography in teaching machines (Fry, 
Bryan and Rigney, 1959) and current trends toward the automating 
of learning and how they affect the Navy's technical training pro- 
blems (Bryan and Rigney, 1959). 

b. Carr, under contract with the Air Force, has presented an 
interesting selected review of the literature with emphasis on those 
studies that provide a functional analysis of teaching machines 
(Carr, 1959). 

c. Four Air Force scientists at the Wright Air Development 
Division have clarified and explained the concept of teaching ma- 
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chines and its implication to military training problems (see arti- 
cle by Eckstrand, et al., in this section). 

d. For the Army, HumRRO has sponsored an annotated bibliog- 
raphy in the automation of instruction (Darby, 1959, and Robinson 
and Fiegel 1961) have published a history, principles, and appli- 
cations of teaching machines for the Army Rocket Guided Missile 
Agency. 

e. A critical and analytical review of the state of the art in 
automated teaching has been prepared by Silverman under a con- 
tract to the Naval Training Device Center (Silverman, 1930b). 

PROGRAMMED INSTRUCTION AND NONVERBAL BEHAVIOR 

Although only passing reference has been made to earlier more 
basic research efforts (they are fully described in the referenced 
reports), they have been instrumental in the formulation of the 
programing and tryouts currently in progress. They have also 
opened up further areas of potential application in programed 
instruction. Among these is an extension of programed instruc- 
tion alluded to in the introduction. The bulk of what has been dis- 
cussed or referenced so far relates to verbal learning. However, 
an important class of skills required by the armed forces can be 
described as perceptual-motor (for example, vehicular control). 
There is a potential application and extension of programed 
techniques to this nonverbal behavior field through what might be 
called "adaptive" trainers. 

The Ohio State University Aviation Psychology Laboratory, 
under U. S. Naval Training Device Center sponsorship, has been 
investigating tracking performance as a function of differ ing sched- 
ules (step sizes) (Briggs, 1961). The trainee practices in one 
tracking configuration until a given level of skill is attained, at 
which point the training device (self tutor) provides him automati- 
cally with a different (more difficult) configuration. The measure 
of success is the extent of complexity achieved in a given time. 
The Engineering Psychology Branch of the Naval Research Labora- 
tory has developed a tracking device with such features. 

A great deal of work remains to be done in this area. For ex- 
ample, which dimensions of the task should respond to trainee 
output and at what points in the training process should changes 
take place? The Office of Naval Research under contract to Dun- 
lap and Associates is exploring the potential utility of such sched- 
uling to problems of personnel selection and equipment design. 

In the case of nonverbal auditory behavior, Bolt, Beranek and 
Newman, under contract to the U. S. Naval Training Device Cen- 
ter, is exploring the application of automated teaching principles 
to training auditory skills. A computerized research tool-has been 
developed for this purpose. 
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OVERVIEW AND SUMMARY 

The scope and intent of this article does not permit a complete 
or detailed review of the past, present and future of military re- 
search and development in programed learning. The authors 
have contented themselves with indicating the trend of present 
development and a listing of representative research programs. 
It is hoped that the reader will have sufficient data to explore more 
thoroughly those programs of particular interest to him. A few 
words are in order, however, concerning future trends. 

It was contended (and, hopefully, demonstrated) that not only 
have the armed forces responded to educational innovation, but 
that they have done so with more alacrity than the field of educa- 
tion in general. In the area of programed teaching, however, 
routine application by the armed forces is proceeding in, if any- 
thing, a somewhat less accelerated fashion than it is in civilian 
quarters. This is due in part to the widespread impact of the 
technique on general education, the limited utility to the military 
of commercially available programs and the consequent burden 
of special program development. 

In addition, there are many gaps in research information con- 
cerning the relative merits of such programed instructional 
features as kinds of programs (linear -branching), responses (con- 
structed-nonconstructed; overt- covert), and influence of various 
kind of subject matter. These gaps even extend to the value of 
automated instruction itself as compared with more conventional 
instruction (Silver man, 196 Ob). 

The supporting research programs of the armed forces will 
continue to attempt to close these gaps but, in part because re- 
search in educational methods is difficult to generalize, an in- 
creasing emphasis will be placed on the development and applica- 
tion of programs. This course will provide information on one of 
the major questions still facing researchers in programed instruc- 
tion. That is, "What are the effects of such instruction when it is 
in long term use? " Will initial novelty effects be replaced by atti- 
tudes less propitious to the learning process? The continued use 
of programed instruction may well bring to light the utility of 
features not evidently important at present. 

The trend in the armed forces is, then, one of cautious opti- 
mism. The research, development and application resulting from 
this trend should benefit both the armed forces 'and education in 
general. 
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Time Saved by a Tryout of Automatic 
Tutoring 

By CLARK L. KOSHER, and JAMES A. NOLAN 
Lackland Air Force Base 



The purpose of this research study was to analyze the results 
from the administration of a self- instructional device compared 
to traditional classroom teaching methods. 

Military students in training in three separate courses were 
selected on a random sampling basis for this experiment. A 
scrambled book was prepared for a designated portion of instruc- 
tion within each course and programmed after the Pressey method 
of the multiple choice problem solving technique. 

Crowder type scrambled books were prepared by educational 
specialists in conjunction with instructors of the experimental 
hours to insure complete agreement of desired learning outcomes. 
A pretest and end- of- instruction test consisting of different items 
measuring the objectives of these hours were prepared and vali- 
dated. 

In each course, two control groups were given normal class- 
room instruction by prescribed lesson plans and two experimen- 
tal groups were given the scrambled books. Experimental groups 
were also subject to any films, student literature, or performance 
projects normally given to the control groups. 

One control group and both experimental groups were given a 
pretest while all groups were given the end-of-instruction test. 
All groups were readministered the end-of-instruction test at 
eleven to thirty-eight days later to measure retention of learning. 

No observable Hawthorne effect was noted for the groups which 
were pretested as compared to the excepted control group. Stu- 
dents learning via the scrambled book finished in much less time 
than was allotted for normal classroom instruction. 

An analysis of results (see Table 3-2-1) showed students in ex- 
perimental groups gained a comparable amount of academic 
learning in less time than students in the control groups. No ma- 
jor differences were evident between groups on the end-of-instruc- 
tion test given to measure the retention of learning outcomes. 

An analysis of student achievement by general aptitude revealed 
no major differences between control and experimental groups. 
Students with higher aptitude gained more than those of lower 
aptitude; however, gains for students of both high and low aptitude 
were greater than in normal classroom instruction. 
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TtME SAVED BY AUTOMATIC TUTORING 
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Analysis of data for the approximate one -third of the students 
in the experimental groups who finished first versus the approxi- 
mate one-third who finished last showed that the slow-pacers 
learned as much as or more than fast-pacers. 

This experiment was restricted in scope and its findings merely 
establish a trend; however, it appears that students learn effec- 
tively in less time by the scrambled book self -instructional de- 
vice. 



A Feasibility Study of a Special Machine - 
Taught Oral-Aural Russian -Language Course 



By EUGENE H. ROCKLYN and RICHARD I. MOREN 
Human Resources Research Office 



Can speaking and understanding a foreign language be effectively 
taught without benefit of a human instructor? An opportunity to 
find the answer to this question arose when we were asked to de- 
velop a special oral-aural language course, based on a limited 
vocabulary, for obtaining tactical information from Russian- 
speaking personnel. 

The effectiveness of this course, as measured by the ability of 
the students to adequately speak and understand the limited Rus- 
sian course material, supports the feasibility of machine -teaching 
limited language courses. The possibility of machine -teaching a 
full-scale foreign language is also supported by this research. 
This concept of limited language, with the general format of ques- 
tion frames and inserts plus classes of the most probable answers, 
could be applied to achieve economy of training time in other sit- 
uations in which there is a need for some, but not complete, for- 
eign language ability. 

THE PROCEDURE 

For purposes of a pilot study to determine the feasibility of 
such a completely automated course, content was selected from 
military literature. This content, first drafted in English, con- 
sisted of 16 commands such as "surrender," "hands up," and 
"speak slowly," 10 question frames such as "have you there...," 
"how many.,./ 7 and "where...;" 100 terms such as "tanks," 
"men," and "guns" that could be meaningfully inserted into most 
of the question frames. In addition, 100 basic answers were com- 
posed of words that might be used by a prisoner in answering these 
tactical questions. These were words such as "yes" and "no," 
cardinal and ordinal numbers, compass headings, time divisions, 
etc., plus the necessary connectives. Answers were formulated 
so that the words used could be interchanged to make up other 
probable answers. The total vocabulary consists of about 450 
words. The course objective was to teach the students to give the 
commands and ask the questions in Russian and then to translate 
the related Russian answers into English. 

Programing the course material was the heart of the re- 
search. The procedure of arranging and sequencing the course 
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material in a manner calculated to duplicate the functions of hu- 
man instructors, maximize learning efficiency, minimize learn- 
ing time, and maintain motivation was based on a review of pre- 
vious research and classroom observation. Programing was 
done in the following five major steps: 

1. The English course material was translated into Russian, 
using the spoken rather than the written language as the major 
guideline. Each utterance was translated to achieve compara- 
bility of word order and equality of word number. 

2. The course material was divided into three sections to 
facilitate mastery of pronunciation by first learning to under- 
stand the material that later was to be spoken, to provide a logi- 
cal framework for exposure to continued repetition of material 
by using the same material in all three sections for different 
purposes, and to permit the use of the student's native tongue 
and, thereby, letting the student enter immediately, directly, and 
successfully into learning a foreign language skill. The first 
section consisted of understanding the questions and commands 
or translating the Russian into English. The second section con- 
sisted of speaking the questions and commands or translating 
English into Russian. The third section consisted of understand- 
ing the answers or translating the Russian into English coupled 
with asking the questions in Russian. The material of each sec- 
tion was programed into lessons using the principles of gradu- 
ated difficulty and meaningfulness of material. Graduated diffi- 
culty was achieved by selecting material based on the amount or 
length of utterance and similarity of the Russian to the English. 
Meaningfulness of material was achieved by presenting the ma- 
terial, for the most part, in the form of a complete command, 
question, or answer. 

3. The lessons were composed of items defined as any com- 
mand, question, or answer or any part thereof. The basic pro- 
graming procedure for the items was derived from the method 
of paired associates and straight translation. Essentially, this 
consisted of a presented stimulus, either Russian or English, a 
response interval during which the student can make a response 
either in English or Russian, and presentation of the correct re- 
sponses, either English or Russian. The items of each lesson 
were programed using the principles of decreasing assistance, 
or cue drop-out, and varied repetition. Cue drop-out was at- 
tained by arranging each lesson in five lists that contained iden- 
tical material in various orders. The first, or direct-cue, list 
completely gave away each answer or response. The second and 
third lists gave progressively fewer cues to each answer. The 
fourth and fifth lists gave no cues to the answers. Varied-repe- 
tition was attained by presenting identical items two or more 



MACHINE TAUGHT ORAL-AURAL RUSSIAN 75 

times in a list or by presenting identical parts of items two or 
more times in a list. 

4. Printed panels prepared for every lesson presented detailed 
instructions, limited samples of list construction and occasionally 
a brief note of linguistic interest. On the opposite side of each 
panel was the score sheet for that lesson. A legal-sized clip 
board lined with corrugated cardboard was used to hold a score 
sheet. The students used metal styluses to punch holes in their 
score sheets to indicate the successful or unsuccessful outcome 
of each of their individual responses in every lesson. 

5. All lessons were recorded on master tapes. Five copies of 
each master tape were made and put into cartridges to make five 
complete nets. There were 82 tapes in one complete set exclud- 
ing seven test tapes. There were 65 lesson tapes, 16 review-pre- 
view tapes, and one pronunciation tape. The major problem in re- 
cording the master tapes was in timing the student response in- 
tervals. These are empirically fixed at a time in which the cor- 
rect response, if known and spoken at the normal rate, could be 
given twice. Dual-track language -teaching tape recorders that 
were modified to handle the tape cartridge were used in conjunc- 
tion with earphones and a microphone attached to a flexible stand. 

COURSE ADMINISTRATION 

This Russian course was given at George Washington Univer- 
sity using five small classrooms. Each of these small rooms con- 
tained a table, chair and electrical outlet. The taped-lesson li- 
brary and a monitor's desk were located in the hall next to these 
rooms. 

The course was administered twice, each time to five students. 
The first administration was a shakedown of equipment and pro- 
cedures. Taking advantage of the shakedown experience, appro- 
priate changes were made and incorporated into the course. The 
second administration was conducted with the students administer- 
ing the course to themselves without interference by the experi- 
menters. What follows concerns only the second administration 
and its results. 

The five students taking this course were enlisted men in the 
U. S. Army who had just completed their basic training. These 
men ranged in age from 18 to 23 and in education from the llth 
grade through college. None had any previous Slavic -language ex- 
perience. Their language -aptitude -test scores, as compared with 
college freshmen, ranged from the 10th to the 65th percentile. 

In the beginning, the students were given a language aptitude 
test and a brief orientation. Then, each man was assigned one of 
the small rooms and specific equipment. After he set up his 
equipment, he read his instructions and study guides, picked up 
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the first tape cartridge lesson with its panel, and started to work. 
When he finished the first lesson, he replaced his recorded car- 
tridge, and put his punched score sheet on the monitor's desk. 
He then selected his next lesson and the process was repeated. 
After the first day the procedures were identical except that each 
morning the student selected a re view -pre view tape that covered 
the material of the preceding day. After going through this tape 
once he returned it to the tape library, took out his next sched- 
uled lesson, and so on. 

There were two methods of keeping track of what the students 
did on each lesson. First, the students' verbal responses or 
answers were recorded on the lesson tapes. If all the items in a 
lesson were answered correctly, the tape was kept as a perma- 
nent record. If some of the items were answered incorrectly, 
the student repeated the lesson, automatically erasing all of the 
old responses and recording his new ones. Lessons were re- 
peated in this way until all the items were answered correctly. 

Second, the student punched a hole in an appropriately marked 
place on the score sheet to designate each response hemade as cor- 
rect, partly correct, or wrong. A non -Russian -speaking member 
of the research staff sat at the monitor's desk. He inspected the 
punched score sheets for evidence of gradual acquisition and 
mastery of the material. This monitor also recorded the time 
the lessons were taken out and returned and any rest pause that 
occurred while the student was working on the lessons. There 
was practically no interaction between the monitor and the stu- 
dents. 

Every student took a self-administered tape test after each 
section and, at the end of the course, a final tape test that covered 
all of the course material. A simulated interrogation test then 
was given, with the students questioning native Russians. 

THE RESULTS 

Working about 6 1/2 hr a day, all five students completed the 
course in about 23 days. Course completion times ranged from 
22 1/2 to 24 days. In the final tape test covering all of the course 
material, the spoken Russian of all five students was first 
scored by a native Russian expert on the research staff. Each 
student was then re-scored on the adequacy of his speaking abil- 
ity by other native Russian experts. These experts, four in num- 
ber, are professors of Russian at the U. S. Navy Language School 
in Washington, D. C. Prior to the time they scored the tape, they 
had never heard the particular student speak Russian. The scor- 
ing procedure consisted of listening to the student's spoken Rus- 
sian and writing down the English translation if the Russian was 
understandable. An over -all evaluation of each student's pronun- 
ciation also was made. 
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Total test scores ranged from 90 to 93% correct. Scores on 
speaking Russian ranged from 86 to 93% of course material cor- 
rectly spoken with evaluations of pronunciation ranging from fair 
to excellent. Scores on understanding the Russian ranged from 
89 to 93% correct. 

The simulated interrogation test consisted of the students 
questioning native Russians. Each student was given a stack of 
cards with an English question on each. He translated the ques- 
tions into Russian, heard the native Russian's answer, and trans- 
lated this answer into English. The native Russian had been given 
a story, written in English, of a fictitious situation. From this 
story he composed the answers he gave in Russian to the student's 
questions. In this simulated questioning situation, the five sol- 
diers obtained from 85 to 97% of the possible information given 
by the Russians in answering the questions. 

DISCUSSION OF THE RESULTS 

All experimentally administered courses that result in favor- 
able acquisition and usability scores must face the criterion of 
producing comparable results when given under non-experimen- 
tal conditions. The overall construction or programing of this 
limited Russian -language course into sections and lessons de- 
finitely structures the learning task, provides reasonable and de- 
finite subgoals, and a clear and achievable terminal point. The 
specific programing of the lessons to assure successful respond- 
ing and continual reinforcement seems to produce and sustain a 
high level of student motivation. 

On the basis of these factors the prediction can be made that 
if this course were administered under nonexperimental condi- 
tions to students of equal aptitude, the results could be favorably 
compared with those recorded in this experiment. Further sup- 
port for this prediction is derived from the fact that the experi- 
mental course was completely automated. This means that the 
precisely standardized lesson material will be presented in ex- 
actly the same fashion in any nonexperimental administration of 
the course. And in addition, replacing language instructors by 
tape recorders makes this standardized lesson material available 
for repetitive practice on a basis that can be achieved only by the 
use of machines. 

* * * 

The research reported here was conducted by the authors while 
they were employed by the George Washington University, Human 
Resources Research Office, operating under contract to the De- 
partment of the Army. Opinions and conclusions are those of the 
authors and do not necessarily represent views of the University 
or the Department of the Army. 
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Air Force Systems Command 



Teaching machines have aroused a great deal of interest re- 
cently in various professional and scientific fields. Appropriately 
enough the military psychologist has been among the most inter- 
ested in this new technology. Training has always been one of the 
factors of vital importance to efficient military operations and in 
recent years has definitely become big business within the mili- 
tary. Within the U. S. Air Force, for example, it is conserva- 
tively estimated that over one billion dollars was spent on train- 
ing operations during the past year. Consequently, military psy- 
chologists have devoted considerable effort to developing techni- 
ques, methods and devices for improving the efficiency of train- 
ing. It is small wonder, therefore, that the teaching machine or 
automated training device has attracted the attention of the mili- 
tary psychologist early in its developmental history and has set 
his imagination racing. For these self-instructional devices hold 
out the promise of solving, or alleviating, a number of the pro- 
blems that have beset military training. 

Already, three names have been used in this paper to refer to 
the topic under discussion teaching machines, automated train- 
ing devices, and self-instructional devices. These are the terms 
most commonly used, but there are others, and this diversity of 
nomenclature reflects both the newness of the field and the variety 
of special meanings which teaching machines have for different 
persons. Regardless of the terminology used, or the special in- 
terests of particular investigators, these devices constitute a 
class of instructional media with unique characteristics which 
differentiate them from other media. Throughout this paper, the 
three terms listed above will be used where each seems most ap- 
propriate. 

THE TEACHING MACHINE 

Teaching machines are devices which provide effective instruc- 
tion to a student or trainee without the direct participation of a 
human instructor. In other words, these devices simulate in one 
manner or another the functions normally fulfilled by a human in- 
structor in individualized teaching or training. Their functions 
can be categorized generally as follows: 
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a. Presentation of information. Some machines present infor- 
mational material by a display which is part of the device itself. 
Other machines depend upon supplementary sources of informa- 
tion^such as textbooks or other adjunctive displays. In either type 
the information presented is the material to be learned and pro- 
vides the background for the next function. 

b. Presenting questions or problems. All self-instructional 
devices present the student with questions or problems to which 
he must make some active response. The questions or problems 
posed are keyed to the information which has been presented to 
the student for mastery. In some devices, functions a and b are 
combined. The information to be communicated is actually con- 
tained in the questions or problems which are presented. This 
purely Socratic method of instruction has been used with con- 
siderable ingenuity in some devices and more will be said about 
this technique later. 

c. Accepting and evaluating responses. As mentioned above, 
self-instructional devices require active responses on the part 
of the learner. In certain cases he selects his response from 
among alternatives which are presented to him, and in other cases 
he must construct his responses. All self-instructional devices 
also include some system for evaluating the student's responses 
against a criterion of adequacy. 

d. Providing reinforcement. Closely associated with the func- 
tion of evaluating the student's response is the function of inform- 
ing the student of the adequacy of the response. This information, 
or knowledge of results, may be indicated by a "right" or ' 'wrong 37 
signal of some kind or may come from the student's own compari- 
son of his constructed response with a standard. In any case, the 
prompt and clear presentation of knowledge of results is a consis- 
tent feature of self -instructional devices. In some cases a distinc- 
tion has been made between knowledge of results and reinforce- 
ment, the argument being that confirming the correctness of a 
learner's responses to problems maybe expected to be reinforc- 
ing only if the learner's motivation is intrinsic to the task being 
learned (Carr, 1959). Self-instructional devices are well suited 

to extrinsic reinforcement also, but this problem has not yet been 
carefully analyzed. 

e. Programing learning. Self-instructional devices present 
material to the student in a manner and sequence intended to pro- 
vide for efficient learning. The preparation and ordering of the 
blocks or "frames" of information to be presented by a teaching 
machine is called programing, and in all but the most complex 
devices, this function is performed by a human. Many self-in- 
structional devices, however, are designed so that they can modify 
their program in a manner indicated to be desirable by the stu- 
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dents' responses to questions or problems. Such a capability is 
called adaptive programing. The degree of adaptiveness among 
devices varies from merely dropping out mastered items in a 
serial program to selecting or generating problem situations 
which are highly tailored to individual needs. 

Self-instructional devices derive their distinctiveness from 
performing all of the above functions in teaching or training a 
student. Other teaching aids, such as books,, charts, films, and 
graphics, merely present information to the student, while more 
complex training devices, such as procedures trainers and simu- 
lators, require the constant presence of one or more instructors 
to fulfill the full range of teaching functions. The teaching ma- 
chine alone provides true self-instruction on a routine basis by 
simulating the functions of the human instructor through electri- 
cal or mechanical means. As such, it provides a unique, new tool 
for exploitation in education and training. 

MILITARY TRAINING PROBLEMS 

Teaching machines have a special attractiveness to the mili- 
tary psychologist because they seem so particularly adapted to 
certain challenging problems in military training. These pro- 
blems and the relationship of self-instructional devices to them 
are discussed briefly below since the value of these new devices 
will have to be assessed in the setting of these practical circum- 
stances. 

Cost of Training 

Military training is expensive. It is costly first of all because 
of the complexity and diversity of the jobs involved and because 
of the large numbers of trainees involved. In fiscal year 1958, 
for example, the Air Training Command of the United States Air 
Force (USAF) gave technical training to approximately 300,000 
men in several thousand different formal and special courses at 
a total operating cost of over $800,000,000. Since the major Air 
Commands also do considerable training, these figures represent 
only a part of the training within the Air Force. But training is 
expensive for another reason. It is expensive because there is 
so little time over which to amortize the cost of training. A 
large number of personnel given initial specialized training by 
the military services are available for only a single enlistment 
(as short as three years in the USAF), and even those who remain 
in the service require frequent retraining due to the constant in- 
troduction of new weapon systems. For these reasons, the ratio 
of training cost to useful work time is high in military organiza- 
tions. There are several ways of reducing this ratio but one of 
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the most direct is to reduce the cost of training. As a conse- 
quence, the military is always in search of ways to increase the 
rapidity and economy of training. Self-instructional devices offer 
a number of interesting possibilities here which can be appre- 
ciated without elaboration; need for fewer instructors, use of 
standardized materials, better and more flexible use of available 
training time, and assurance that efficient training techniques 
are being used. 

Individual Differences 

Individual differences in ability to profit from instruction have 
always been a perplexing problem in education by anything other 
than a tutorial system. In military training, there is a problem 
of almost equal magnitude in spite of selection standards and ap- 
titude testing. This problem, although by no means unique to the 
military, is accentuated by the requirement for extremely rapid 
training of large numbers of trainees. Self -instructional devices 
seem ideally suited to help alleviate this problem. These devices 
offer many of the advantages of individual tutoring without the at- 
tendant disadvantages of high cost, shortage of skilled instruc- 
tors, etc. Automatic tutoring provides instruction paced to the 
needs of the individual student and can be adjusted to the trainee 
as he is at the specific moment. In this manner, each student 
moves through the instructional program as rapidly as his talents 
allow him. 

Quality Control 

Military training requires rigid quality control of the products 
of training. Normally, the only purpose of military training is to 
develop specific human performance required in various military 
jobs. The specific performance must be produced with high reli- 
ability if a military operational capability is to be assured. Once 
again, the characteristics of teaching machines give cause for 
optimism with regard to this problem. Because teaching machines 
constantly measure and evaluate the trainee's performance in 
using what he has been taught, the teaching machine assures the 
proper level of all skills and knowledges that are required. Since 
training ceases only when each trainee has successfully dealt 
with each block of instructional information, a uniform and highly 
predictable product (i.e., trained man) is produced. Hence, for 
operational planning, much greater reliance may be placed on ob- 
taining personnel with the required performance capability. 
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Shortage of Instructors 

The shortage of competent, experienced instructors is one of 
the most serious problems in military training. The etiology of 
this problem is complex, involving administrative factors and 
military tradition, as well as aptitudes of available personnel. 
Nonetheless, the problem exists and is a serious source of dif- 
ficulty in most attempts to increase the effectiveness of military 
training. The self -instructional device is a natural solution to 
this problem and hardly requires elaboration. The use of auto- 
mated training devices reduces the need for human instructors 
considerably. Because instructional material is centrally pre- 
pared, requirements for competence in those "instructors" 
which are needed in the field is also reduced. Therefore, such 
instructors as may be necessary will function more as proctors 
and will themselves need less aptitude and less formal training. 
Thus, a major barrier to improved military training is elimi- 
nated through the use of automated training devices. 

Trainee Motivation 

Motivation is frequently a special problem in the military 
training situation. This is particularly true where the trainee is 
merely fulfilling an obligation for military service and has no 
career intentions with respect to military service. These people, 
moreover, usually constitute a significant proportion of the man- 
power available, and it is incumbent upon the military to use them 
as efficiently as possible. In the absence of career motivation, 
special attention must be given to making training an interesting 
and rewarding experience. Although there is not much firm data 
regarding the motivational characteristics of teaching machines, 
experience to date indicates that they are well received by 
trainees and that this positive reaction persists over substantial 
periods of time. More attention will be given to this problem 
later. 

These are by no means all of the military training problems to 
which machines are germane. Nevertheless, the discussion 
should be adequate to explain the interest and enthusiasm of mili- 
tary psychologists for this new technology. 

So far, our discussion of automated training has been intention- 
ally at a very general level. In the remainder of the paper, how- 
ever, various aspects of this topic will be discussed with some- 
what greater specificity. First, some representative automated 
training devices and materials will be described and briefly dis- 
cussed. Next, some current problems and areas of investigation 
will be summarized and special attention will be given to some 
projects which have special relevance to the military uses of 
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teaching machines. Lastly, some discussion will be devoted to 
possible uses of automated training devices in the military and 
to certain requirements which need to be met before these de- 
vices can be fully and effectively exploited. 

SOME REPRESENTATIVE TEACHING MACHINES 

A few years ago it would have been possible to describe vir- 
tually the entire population of teaching machines in the space we 
are now devoting to a "representative" sample. In fact, the pro- 
liferation of machines during the past two to three years has been 
so great that the "representativeness' 7 of our sample probably 
is somewhat dubious. However, the machines selected for dis- 
cussion do represent some of the major attempts to mechanize 
the instructional process. 

Teaching- Ma chine Design and Programing Philosophy 

In general, the term "teaching machine" is a misnomer. 
Most machines do not teach, they merely provide vehicles for 
the presentation of preprogramed materials in a fashion which 
simulates some of the functions of an individual tutor. Thus, the 
characteristics of specific machines depend not only on the nature 
of the skills or knowledges to be learned, but also upon the par- 
ticular programing philosophy to be implemented. 

At the present time, there .appear to be three major approaches 
to the programing of verbal and symbolic materials. The first is 
the simple multiple -choice approach originated by Pressey 
(1926, 1927). This is simply a variant of the usual objective ex- 
amination augmented by self -scoring features, which provide the 
student with immediate feedback concerning the correctness of 
his responses. Pressey did not intend that his self-instructional 
tests and associated machines replace textbooks and other infor- 
mation presenting aids. Rather, they are designed to complement 
the conventional techniques by identifying areas of weakness and 
guiding the student in remedial study and review. Pressey's ma- 
jor contribution to teaching machine technology is his early rec- 
ognition of the value of mechanizing some aspects of the train- 
ing process. 

The second major approach to programing is that developed 
by Skinner (1958) and based in part, at least, on principles de- 
rived from his conditioning work with animals. His teaching ma- 
chine presents the material to be learned in a fixed sequence 
and requires that the student's responses be constructed rather 
than selected from a multiple choice item. Another relevant 
feature is the "information distance" between successive steps 
in the program; it is so small that correct responding throughout 
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acquisition is virtually assured. In addition to Skinner's contri- 
butions to the art of programing, he is largely responsible for 
the recent surge of interest in machine instruction. 

The third major approach to the programing problem is the 
variable sequence, multiple choice technique developed by Crow- 
der, who labeled his approach "intrinsic programing" (1959). 
His materials consist of units of information (usually a paragraph 
or less in length) followed by a multiple choice question related 
to the particular unit being presented. This approach differs 
from those involving fixed sequences, in that student's response 
to a given question determines the material that he sees next. 
Thus, if he answers a question correctly he is presented the next 
unit in the main sequence. On the other hand, if he makes an er- 
ror, he is shunted to a sub-sequence which permits him to review 
the previous material and explains the nature of his error. Thus, 
the amount of material to which a student is exposed is automati- 
cally tailored to his ability. This type of programing, like many 
others, does not require a machine for presentation. In fact, one 
of its most popular applications is a printed form called a 
"scrambled book." A scrambled book differs from the conven- 
tional book in that although the pages are numbered consecutively 
they are not read in the usual order. That is, after the student 
selects a particular multiple choice alternative he then turns to 
the page whose number is listed next to his choice. If he was 
correct, the material on the page confirms his previous response 
and presents the next unit of information. If he was incorrect, the 
information on the page is designed to clear up his misunderstand- 
ing before returning him to the previous page to try again. Crow- 
der's machines, to be described later, provide more flexible 
semiautomatic mechanisms for storing and "turning the pages" 
of intrinsically programed materials. 

To date, the major approach to the programing of machines for 
psychomotor skill teaching is that proposed by Pask (1957, 1958). 
Pask's approach is unique in that the teaching program is not 
completely predetermined. Rather, the machine is designed to 
emulate an ideal instructor who is in continual interaction with 
the trainee. It is assumed that the machine instructor and the 
trainee are engaged in a partly competitive, partly cooperative 
game (Pask, 1958). At the outset of training the machine has very 
few built-in rules. The rules are developed during the course of 
play as the machine learns about the trainee and builds a model 
of his behavior which is used to adapt the training routine to his 
individual requirements. The basic control mechanism for ac- 
complishing the tutorial functions in Pask's machines is a self- 
organizing computer that can be adapted to the teaching of a 
wide variety of job skills. 
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Pressey's Machines 

Pressey is generally conceded to be the father of the teaching 
machine. In 1926, he described "a simple apparatus which auto- 
matically gives and scores a test, and which will also, automati- 
cally, teach and teach informational and drill material more ef- 
ficiently, in certain respects, than the human machine" (Pressey, 
1926). This early device, similar to that shown in Fig. 3-4-1, 
can be considered the prototype for the multiple choice approach 




Fig. 3-4-1 One of Pressey's early machines 

to machine instruction. Basically, the device consists of a win- 
dow in which multiple choice or true -false items are presented, 
a set of four numbered response keys which the student presses 
to indicate his answer, and a counter for tallying errors. The 
device has two modes of operationa test mode and a learning 
mode. In the test mode the student is permitted to make only one 
response to any given item. After each response a new item is 
presented. In this mode no indication of response adequacy is 
provided. In the learning mode, on the other hand, an item re- 
mains in the display window until the correct button is pressed. 
Only then does the next item appear. Thus, in this mode the stu- 
dent is given knowledge of results after each response. This de- 
vice also possesses the capability of dispensing a candy reward 
when certain goals are met. A later machine designed by Pressey 
(Pressey, 1927) also incorporates a "drop out" feature. That is, 
the machine can be set to sequence through a given set of items 
one at a time until each item has been answered correctly on two, 
three, or four successive trials. As the selected criterion is 
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achieved for an item, that particular item is "dropped out" on 
subsequent trials. 

Skinner's Machine 

Most of Skinner's machines require that the student compose 
or construct a response rather than merely select one (ref. 17). 
The simple disk machine in Fig. 3-4-2 is probably the best known 
of Skinner's devices. In this machine, material is printed on 30 
radial frames of a 12 -in. disk. Each frame consists of a question 




Fig. 3-4-2 Skinner's disk machine 

or statement plus a correct answer. During the course of teach- 
ing, the statement only is exposed in the lower portion of a dis- 
play window. The student responds by writing his answer on a 
paper strip beneath a second opening. He then operates a lever 
on the front of the machine which moves his response under a 
transparent shield and uncovers the correct response in the dis- 
play window. If his response corresponds to the correct one he 
moves the lever horizontally. This punches a hole in the answer 
sheet opposite his response, recording his judgment of "correct," 
and modifies the machine so that this item will be skipped during 
subsequent presentations of the program. He then returns the 
lever to the starting position and a new frame is presented. The 
student continues to work through the program until all items 
have been answered correctly. When this occurs the disk revolves 
without stopping and the assignment is finished. Other constructed 
response devices have been designed in which the student com- 
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poses his answer using a set of alphabetic and/or numeric sliders. 
The student's answer is compared by the machine with a coded 
response. If correct, the machine automatically presents the next 
frame. If incorrect, the student's response is cleared and he 
must compose another. The machine will not advance to a second 
frame until the first has been answered correctly. Not all of 
Skinner's devices require response construction, however. He 
has recently been experimenting with a series of preverbal multi- 
ple choice machines for use with young children (Corrigan, 1959). 

The Subject Matter Trainer 

The Subject Matter Trainer was developed by the USAF to pro- 
vide an automatic means for teaching and testing certain kinds 
of technical information without the aid of an instructor (Besnard, 
et al., 1955a). This device has many features in common with 
Pressey's earlier machines. Each of 20 stimulus items is pre- 
sented one at a time in a display window located on the left side 
of the panel, shown in Fig. 3-4-3. The 20 response items are 
mounted in a scrambled order on a plate at the right side of the 
panel. Coded answer plates are readily interchangeable to per- 
mit new orders for correct response items. Each response item 
is located between a response button and a green indicator light. 
The device has two digital counters for recording student perform- 
ance. One counter advances one unit for each response made 
by the student while the other tallies only correct responses. A 
later model of the trainer also incorporates a "drop out" feature 
similar to that described for Pressey's machine above (Besnard, 
et. al., 1955b). The following six modes of practice are provided 
by the trainer: 

a. Quiz mode. The name of this mode is somewhat misleading, 
since it is the machine and not the student that is being "quizzed." 
When a stimulus item appears in the window the student presses 

a "quiz" button which illuminates the green light next to the cor- 
rect answer. Thus, the machine guides the student through the 
correct response sequence for the entire series of stimulus items. 

b. Modified quiz mode. In this mode the "quiz" button is de- 
activated. When a stimulus item appears, the student is required 
to press the response button next to his chosen answer. If his 
selection is correct the green light next to his answer will glow, 
and the machine can be advanced to the next stimulus item. If he 
is incorrect, however, a red light glows and a buzzer sounds indi- 
cating an error. Simultaneously, the green light next to the cor- 
rect buzzer sounds indicating an error. Simultaneously, the 
green light next to the correct answer comes on telling him what 
his answer should have been. The student then must depress the 
correct answer button before the machine can be advanced. 
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Fig. 3-4-3 The subject-matter trainer 

c. Practice mode. This mode is like the previous one except 
that the student must continue pressing buttons until the correct 
answer is found. When the correct answer finally is selected, the 
corresponding green light will glow and the machine can be ad- 
vanced to the next stimulus item. 

d. Single try mode. In this mode the student is permitted only 
one'response for each stimulus item. After each response he re- 
ceives either a red or green light. 

e. Paced practice mode. This mode is primarily for experi- 
mental use. It permits duplication of the "practice' ' and "single 
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try" modes by an experimenter who controls the rate of appear- 
ance of new items. 

f. Test mode. In this mode the student receives no feedback 
concerning the adequacy of his response. He is permitted only 
one response to each stimulus item before being required to go 
on to the next stimulus item . 

Crowder's Machines 

Two versions of automatic, random access, microfilm projec- 
tors have been developed for use with Crowder's intrinsically 
programed materials. An early prototype was developed a num- 
ber of years ago by the USAF. This first version was called the 
Multipurpose Instructional Problem Storage Device, or MIPS. 
Recently, a much improved device was designed and fabricated 
under Crowder's direction at the Western Design Division of 
U. S. Industries, Santa Barbara, California (Crowder, 1959). This 
latest model, which is shown in Fig. 3-4-4, is known as the West- 
ern Design Autotutor. The program in the Autotutor is stored on 




Fig. 3-4-4 The Western Design Autotutor 

35-mm film, and both still and motion picture material maybe 
intermixed. The advertised storage capacity of the device is 
10,000 35-mm frames, although with a thinner film base this may 
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be increased to 30,000. In use, the student is presented with a 
unit of information on the viewing screen, along with a multiple 
choice question based on that unit. He then selects the next image 
to be presented by entering the number associated with the re- 
sponse he chooses into the selector buttons of the Autotutor. He 
next presses a "View" button, and the device automatically lo- 
cates and projects the selected image. He then proceeds on 
through the program in the manner described previously. The 
machine can automatically record and print out on paper tape 
both the sequence of images viewed by the student and the time 
spent viewing each image, if such information is desired. 

Pask's Machines 

The most publicized of Pask's machines is the Solartron Auto- 
matic Keyboard Instructor or "SAKI" (Pask, 1958). This device 
was designed to teach the operation of a keypunch mechanism such 
as that used for punching IBM cards. As can be seen in Fig. 3-4-5, 




Fig. 3-4-5 The Solartron Automatic Keyboard Instructor (SAKI) 

the major components of the system are a display unit, a key- 
board unit, and a small self-organizing computer which controls 
the training routine and records errors. The display unit consists 
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of a back lighted stimulus card with four lines of exercise ma- 
terial and an analog keyboard. Initially, the item to be punched 
by the student is indicated by illuminating both the appropriate 
figure on the stimulus card and the corresponding key position on 
the analog keyboard. During the early stages of learning the rate 
of presentation of stimulus items is relatively slow. However, as 
the student gains proficiency with respect to speed and accuracy, 
the rate of presentation of stimulus items progressively increases 
to a preselected maximum and the illumination of the keys on the 
analog panel gradually decreases to zero. Throughout training, 
the computer continuously adapts the stimulus program to the 
unique characteristics of the individual trainee. Other versions 
of this device are being developed for use with adding machine 
and typewriter keyboards. In addition to keyboard training, Pask 
has also investigated the use of a self -organizing computer in 
conjunction with a complex radar simulator for teaching pilots 
the tactical and maneuvering skills required in airborne inter- 
ceptions (Pask, 1957). In this application, the computer 
(EUCRATES Mk 1) served the function of a highly skilled instruc- 
tor in adapting training exercises to the particular needs of indi- 
vidual trainees. 

PROBLEMS AND APPROACHES 

On first impression it might seem that the diverse array of 
self-instructional schemes and devices would present diverse prob- 
lems requiring an immense series of highly specific investiga- 
tions. And, indeed, the first research on teaching machines was 
directed at some of the more specific and obvious issues. The 
initial research tended to investigate problems such as, whether 
to require a constructed response as suggested by Skinner (1958), 
or whether to allow a choice among multiple response alterna- 
tives; whether to present but a single series of items to everyone, 
or whether to have branching programs such as those used by 
Crowder (1959); whether to sequence the instructional items in 
such small "steps" that error responses would be virtually im- 
possible,, and so forth. Although some investigations of this type 
are necessary, a more systematic approach is desirable and 
seems to be emerging. Systematic programs of research should 
assure generalization of research findings to a wide variety of 
devices and materials to be learned, including both verbal and 
nonverbal tasks. 

To maintain proper perspective about teaching machines, two 
facts must be recognized. First, teaching machines have no spe- 
cial or unique status in psychology. They are merely a specific 
example of the dynamics of teaching. The second fact to remem- 
ber is that various authors and investigators apply their diverse 
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and previously formulated viewpoints or scientific approaches to 
considerations of teaching machines. Because different authors 
have different conceptualizations of the teaching process, and 
even different philosophies of science, their approaches to teach- 
ing machines and their research are different, not to mention dif- 
ferences in terminology. Awareness of the basis for certain ap- 
parent differences should provide some perspective by which to 
understand and interrelate diverse sources of information about 
teaching machines. 

Skinner's (1958) approach to teaching machines, and pertinent 
research influenced by his approach, quite naturally is consistent 
with his general approach to behavioral science. The research is 
pragmatic. Much of it consists of evaluating the effects of "pro- 
grams" (sequences of instructional questions) prepared in accord- 
ance with a number of previously formulated principles. Some- 
times the research implies as much about the ability of the in- 
vestigator to practice the "art of programing'' as it does about 
the principles of programing or any other conditions of the in- 
vestigation. 

A different approach to teaching machines is illustrated by 
Silberman and Coulson. Although they have not so stated, their 
approach, which emphasizes the statistical interrelationships 
among items in a program, is related to mental test theory. Gagne 
and Bolles (1959) imply a still different approach to automated 
training. They conclude that the same factors govern the effi- 
ciency of learning whether the learning is controlled by a ma- 
chine or by any other means. However, they also point out that, 
compared to traditional research on the influence of specific vari- 
ables on the learning process, research on teaching machines is 
characterized by a much more intense search for the conditions 
that maximize the efficiency of learning. 

Pask (1958) approaches teaching machines from the viewpoint 
of cybernetics and game theory. He conceives of the student and 
the instructor as two self -organizing systems engaged in a game 
which is partly competitive and partly cooperative. Pask, Carr 
(1959), and probably many others, including the present authors, 
believe that both the use of and research upon teaching machines 
must be guided by a functional analysis of the interactions among 
the learner, the material to be learned, and any type of teaching 
guide or "instructor/' The "instructor" may be solely human, 
both human and machine, or solely machine. However, even a 
common point~of-departure such as a functional analysis of the 
instructional process will not counteract the effects of different 
theoretical or systematic viewpoints. We may expect diverse pro- 
grams of research on teaching machines for many years to come. 
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SOME MILITARY USES OF AUTOMATED TRAINING DEVICES 

So far in this article we have commented on the general rela- 
tionship of automated training to military psychology, described 
some representative devices and discussed some current re- 
search issues and projects. Undoubtedly this material has al- 
ready suggested numerous possibilities for applying automated 
training devices in the field of military training. On first im- 
pression, it would appear that there is no problem at all in think- 
ing up all sorts of uses for these devices. Indeed, whenever an 
instructor is involved in a program of training, we have a poten- 
tial for automating the functions of this instructor. But perhaps 
we are moving too fast; potential does not necessarily mean pay- 
off. Contrary to much popular thinking, it is not always techni- 
cally desirable to substitute machines for men. Rather, the 
assignment of functions to man and machines in a system should 
be based upon a careful analysis of the capabilities and limitations 
of each in relation to the functions that are required to produce 
the desired system output and, this is true regardless of whether 
the desired system output is lead pencils, bombs on target, or 
trained personnel. As Bryan and Rigney (1959) so nicely put it 
in their discussion of automated tutoring, " T The machines will 
save us' fallacy is so firmly rooted in current thinking that it 
may not be recognized in this new guide. ?> 

In other words, any discussion of the uses of automated train- 
ing devices in military training (or any other training or educa- 
tional program, for that matter) which attempts to become spe- 
cific, must immediately come to grips with the following two prob- 
lems: defining the role of the instructor in training programs 
of various types and assessing the comparative abilities of hu- 
man instructors and machines in fulfilling these roles. These 
problems have not yet been carefully analyzed and studied. How- 
ever, McKeachie (1959) has pointed out that teaching machines 
have focused the interest of teachers and research workers alike 
on these problems, and that the results of such interest may well 
improve human as well as automated instruction. Indeed, he feels 
that this may be one of the major benefits to be derived from the 
current interest in automated tutoring. 

When a Machine Can Profitably Fulfill Instructor Functions 

Rather than attempt a detailed analysis of the functions of the 
instructor, and then compare the ability of men and machines to 
fulfill these functions, we will discuss the military uses of auto- 
mated training devices in certain extreme classes of instruc- 
tional situations. Our discussion will be limited to cases where 
the instructor functions undoubtedly could be assigned with pro- 
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fit to a machine because the functions are too routine, the func- 
tions are too complex, or no human instructor is available. 

Use of Machine When Instructor Functions Are Too Routine. 
In the first case, we are dealing with situations where it is ad- 
vantageous to use automated training devices because the instruc- 
tional situation is highly standard and routine. It will be apparent 
immediately that these are the same conditions under which it 
usually is profitable to introduce automation in industrial manu- 
facturing operations. A little reflection will suffice to determine 
that a substantial amount of military training meets these criteria. 
Facts, nomenclature, and procedures must be memorized. Con- 
cepts and principles must be mastered. Data flow and component 
locations and interrelationships must be understood. A signifi- 
cant proportion of the time required for training in a military 
specialty usually is devoted to developing an understanding of a 
technical area, to serve as a basis for later skilled performance. 
In such cases the materials to be mastered are specifiable in ad- 
vance and the criterion of mastery is unambiguous. The instruc- 
tional task is that of presenting the material in a manner that 
will insure efficient mastery. Traditionally, this is done by hav- 
ing students sit in classrooms and listen to carefully outlined 
and frequently memorized lectures, study textbooks and manuals, 
participate in occasional recitations and drills, and infrequently 
take examinations. Delivering "canned" lectures, listening to 
recitations and drills and grading objective tests is a patently in- 
efficient use of human instructors. In the first place, instructor 
personnel can be used more efficiently for work which cannot be 
done by machines and which requires more of their unique capa- 
bilities (e.g., determining what information the student needs to 
be taught and programing this material for effective learning). 
In the second place, inexpensive, automated devices can be de- 
signed which can probably out -do the human instructor in accom- 
plishing the routine teaching functions mentioned above. Normal 
classroom instruction is heavily dependent upon passive listening 
and provides little opportunity for active response on the part of 
the student. It provides meager feedback to the student concern- 
ing his progress and relatively little chance for adjusting the pace 
to each individual's capabilities. The teaching machine, on the 
other hand, presents material in small amounts, provides for ac- 
tive response by quizzing often, delivers continuous feedback and 
adapts the instructional pace to the individual student In short, 
the teaching machine seems almost ideally suited to this routine 
type of instruction. There will be problems in their use, of 
course, such as educating training personnel in their proper utili- 
zation, preparing subject-matter programs for the machines and 
integrating them into the existing training frame -work and con- 
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cepts. Nevertheless, it can be said with some confidence that the 
effort will be well repaid in improved training and more effective 
use of available instructional personnel. 

Use of Machine When Instructor Functions Are Too Complex. 
Sometimes, we are dealing with situations where the instructional 
task is quite the opposite from that discussed above; situations 
where the instructional task is too complex or too difficult for ef- 
fective accomplishment by human instructors. Here again we 
have a counterpart in the industrial situation, where many com- 
plex data processing operations are handled by computers with 
a great consequent increase in efficiency. If it can be truly said 
that much military training involves the routine acquisition of 
knowledge and concepts, it is just as true that much military 
training involves the mastery of highly complex, extremely so- 
phisticated skills both on the part of individuals and teams. Hu- 
mans must be trained to monitor and control highly complicated 
machinery such as supersonic aircraft and aerospace systems. 
People must be trained to assimilate large amounts of informa- 
tion from diverse sources and make rapid decisions, as in mis- 
sile count-downs. Teams of individuals must be trained to per- 
form in an integrated manner in large data processing systems 
such as the USAF SAGE system. The skills involved in such op- 
erations are difficult to master and typically require long periods 
of intensive training involving much practice in realistic job en- 
vironments. Historically, such practice was acquired in the job 
environment itself with an instructor present both for tutorial 
and safety purposes. Recent military systems, however, have 
been far too demanding on human skill to permit this approach 
completely; therefore, complex, ground simulators have come 
into vogue to aid in providing both transition and maintenance-of- 
proficiency training. As weapon systems have increased in com- 
plexity, so too have the training simulators, and these modern 
devices now have complex instructor-operator stations and re- 
quire teams of instructors and trainer operators in order to pro- 
vide realistic training sessions. Even so, the requirements for 
coordinating, controlling and scheduling rapidly changing inputs, 
monitoring the complex response outputs of students, and provid- 
ing for realistic interaction between the two are rapidly outstrip- 
ping the capabilities of human instructors. Likewise, the require- 
ments for rapid recording and analysis of complex, multivariate 
events is increasing, and it is upon this function that providing 
knowledge of results to the trainees and programing future in- 
structional exercises is dependent. All of these facts point to the 
conclusion that there is a class of military training problems 
where the role of the instructor must be taken over to some de- 
gree by automatic equipment. Here, of course, we are dealing 
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with quite a different type of automated training device from the 
relatively simple, inexpensive teaching machines used in pre- 
senting classroom instruction, but the concepts and rationale are 
the same. We are discussing an automated or self -instructional 
simulator. In such a simulator, the machine, in addition to its 
usual role of providing system simulation, takes on the functions 
of automatically recording and evaluating trainee performance 
and of adaptively programing subsequent events to maximize the 
trainee's approach to criterion performance. Computers, with 
their capacity for short term storage of huge amounts of informa- 
tion, rapid search and rapid computation are well suited to per- 
form these functions which humans perform quite poorly. With 
machines performing these functions, high quality instructional 
personnel can be used entirely for tasks such as determining what 
aspects of human performance to record, formulating criteria of 
performance adequacy for various stages of training, and develop- 
ing programs of problems for the simulator. These are things 
which the machine cannot do. Thus, we have both men and ma- 
chines doing those things which they do best and, once again, im- 
proved training should be the result. Automated simulators as 
operational training tools are still somewhat in the future and 
considerable development work remains to be done, but the Eu- 
crates device described in Section II has proved that the concept 
is definitely feasible and necessity should soon force the issue. 

Use of Machine When No Human Instructor Is Available. In 
some situations automated training devices are desirable for the 
simple reason that no human instructional personnel are avail- 
able. In such cases, if instruction is to be carried on at all, some 
type of self-instruction is required. There are a number of mili- 
tary situations where this is true. Some examples are remote or 
isolated early warning radar or missile -launch sites aboard 
naval vessels and, in the near future, aboard orbital or space ve- 
hicles. In all of these examples, of course, the type of training 
required is not initial training but maintenance of proficiency 
training. Individuals assigned to these military situations pre- 
sumably would have been well trained in formal training courses 
prior to their assignment, but provision must be made for main- 
taining their proficiency on the job. This problem of maintaining 
proficiency will become more and more critical in the military 
as the expense and complexity of weapon systems increases, 
thereby reducing the opportunity for day-to-day practice of criti- 
cal job skills. It seems obvious that missiles cannot be fired of- 
ten for training purposes nor space vehicles launched to provide 
an opportunity for the crew to practice some particularly diffi- 
cult portion of the mission. 

Maintenance of proficiency training in military settings which 
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are cut off from access to formal instructional facilities poses 
special problems, and it is in this area where automated train- 
ing devices should prove very useful. Books and manuals have 
been used effectively for many years but they generally are of 
limited value in training for jobs which demand rapid decision 
making and/or perceptual-motor skills. In these areas, some 
type of automated training device would seem indicated. Types 
of devices which would prove useful pretty well cover the gamut 
of those discussed thus far and no further descriptions of devices 
will be made here. Rather, two brief examples of situations re- 
quiring automated training devices will be presented. 

Remote, isolated early-warning radar sites involve a great 
deal of complex electronic equipment which must be maintained 
by the small complement of personnel stationed at the site. 
Since continuous operation of the radar screen is vital, prompt 
identification, isolation and repair of malfunctions is a neces- 
sity. To accomplish this, the ability of maintenance personnel 
to troubleshoot all types of possible malfunctions must be main- 
tained at a high level. On-the-job experience cannot be relied 
upon to accomplish this, since the actual occurrence of malfunc- 
tions is fortuitous. What is needed are devices which can auto- 
matically present the maintenance man with a wide range of 
troubleshooting problems, assess his proficiency, and concen- 
trate subsequent training in areas of weakness. Only in this way 
can the continued proficiency of the maintenance man in all as- 
pects of his job be assured throughout his tour of duty. 

Long duration missions, such as those involving interplanetary 
travel or extended orbital flight, could present some serious 
problems with respect to the maintenance of little used, but crit- 
ical behaviors (Eckstrand and Rockway, 1960). For instance, 
according to present calculations, it would be several months 
from launch until a manually controlled soft landing on Mars 
could be effected. In the interim, there would be no opportun- 
ity for on-the-job practice. Thus, this extremely critical ma- 
neuver would have to be performed without even the benefit of 
warm-up. In the absence of firm research data to indicate that 
this particular skill is sufficiently resistant to forgetting, it 
would seem highly desirable to provide an on-board capability 
for maintenance of proficiency training. The same argument, 
of course, could be made for inflight maintenance tasks on 
long duration space missions. 

MILITARY EFFORT NEEDED 

As we have seen, there is no dearth of possible uses for auto- 
mated training devices in military training. But listing possi- 



98 MILITARY PROBLEMS 

ble applications for some new advance in the state-of-the-art is 
quite a different problem from assuring implementation. The 
history of military psychology, at least in the United States, is 
rife with examples of failure to achieve rapid and effective utili- 
zation of new techniques, methods, and devices. Although some 
of the blame for this may be attributed properly to the strength 
of tradition in the military, some must also be laid at the door 
of the military psychologist. We have perhaps not always been 
the best salesmen. It is entirely appropriate, therefore, that be- 
fore concluding this paper we consider the problem of what needs 
to be done by military psychologists to justify the current enthu- 
siasm for teaching machines and to insure their effective exploi- 
tation by the military. 

To Assure Proper Use of Teaching Machines 

Currently there are three areas in which military psychologists 
need to program their efforts to increase the probability that the 
potential of teaching machines will be recognized and used in 
military operations. These three areas are discussed briefly be- 
low. 

Demonstrations of Effectiveness. Demonstrations of the effec- 
tiveness of automated trainers in realistic military training situa- 
tions are needed to obtain required military support and to justify 
further interest in this field. Too often we have failed to realize 
the value of demonstration in military psychology and have neg- 
lected this important step. In our enthusiasm to put our ideas to 
work, we have often put them forward before testing the concept. 
This is not necessarily bad if the ideas have a great deal of logi- 
cal validity, but the chances of getting our ideas accepted are re- 
duced if their feasibility has not been demonstrated. In our eager- 
ness to begin programmatic research on a concept or idea, we of- 
ten have asked for support without giving hard-headed evidence 
of why such support is justified. We argue, with some validity, 
that research is required to identify major variables, their ef- 
fects and their interactions before demonstrations can be accom- 
plished. But, often valid demonstrations are possible long before 
completion of all of the detailed experimentation required to opti- 
mize conditions. In the teaching machine area we should strive 
to avoid committing either the error of pushing utilization with- 
out caution or of exercising too much caution by hiding our can- 
dle under a bushel of long-term research programs. 

Such attempts to demonstrate effectiveness are not without haz- 
ard, of course. There is always the chance of failure due to a 
poor choice of conditions, and it would indeed be unfortunate if 
the real potential of automated training were beclouded by an 
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early failure. As Bryan and Rigney (1959) have pointed out, there 
is also a danger that successful demonstrations will force a pre- 
mature crystallization of teaching machine concepts and proce- 
dures. This would be quite unfortunate and certainly must be 
resisted. Nevertheless, the potential gains seem greater than the 
pitfalls, and efforts to demonstrate the effectiveness of teaching 
machines in military training situations certainly should be sup- 
ported. Some attempts are currently being made to accomplish 
this and these efforts are a good start toward putting the pro- 
gram on a firm foundation. 

Imaginative Planning For Use. Assuming successful demon- 
strations of effectiveness, the military psychologist then is faced 
with the task of assisting, and in many cases leading, in planning 
for the use of automated training devices both in existing train- 
ing programs and in programs which are being developed to sup- 
port new weapon systems. This will require imaginative and 
careful planning. No one or two successful demonstration studies 
will indicate where and when to push for the introduction of auto- 
mated training devices. Yet, if history is any guide, the danger 
is that we will not use all that we know; rather than that we will 
go further than our knowledge permits. Therefore, it seems ap- 
propriate to recommend that military psychologists use all means 
at their disposal to assure that automated training technology is 
applied in all areas of military training where it appears to offer 
significant advantages over current practices. This will require 
salesmanship as well as science, but military psychologists must 
learn to mix these liberally to be effective, 

Systematic Applied Research Program. We have a significant 
amount of information about teaching machines at the present 
time and can recommend their use with considerable confidence 
in a number of military situations. Nevertheless, if these de- 
vices are to be fully exploited, one of our primary needs is for 
an aggressive, systematic applied research program to further 
advance our knowledge in this area. If present trends continue, 
there seems little doubt that this need will be fulfilled without any 
special urging. Research in the area of teaching machines, after 
a number of rather lean years, is now increasing by leaps and 
bounds. Fig. 3-4-6, which is reproduced from Bryan and Rigney 
(1959), presents the trend over the past dozen years. 

There seems little need, therefore, to make the usual plea for 
more research. Many competent investigators are busy identify- 
ing relevant problem areas and conducting programmatic research. 
We can expect with some confidence that much valuable informa- 
tion will be generated within the next few years. 

It would be pleasant to end this discussion on such a reassur- 
ing note, but to do so would give the military psychologist a false 
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Fig. 3-4-6 Frequency per year of teaching -machine studies from 1948 
to September 1959. There were six references prior to 1948. 

sense of security. There are a number of problem areas which 
are closely related to the military uses of teaching machines 
which require study by the military psychologist before any large 
scale use of teaching machines can be recommended. Most of 
these problems are being neglected by current investigators. 
Some of them may not even qualify as research in the usual sense 
of the word, but their importance is not thereby decreased. Some 
of the more important of these problems are the following: 

a. Motivational aspects of automated training. Trainee motiva- 
tion has been discussed earlier as one of the challenging problems 
in military training to which automated training devices might 
make a contribution. Yet, little is known at present about the long- 
term motivational properties of these devices, and an answer to 
this question is basic to their large-scale use. Great teachers 
have long been known for their ability to motivate and inspire 
their students and there probably will be no such great teachers 
among the teaching machines. On the other hand, there probably 
will not be any really bad ones either. Teaching machines can be 
designed so as to present instructional materials in an efficient 



TEACHING MACHINES IN THE MODERN MILITARY 101 

way. What we need now are some studies as to how they can be 
designed so that the student's response to them will be positive 
both initially and on a continuing basis. 

b. Support requirements of automated training. The types of 
facilities, services, and personnel support required to operate 
conventional training programs have been well worked out through 
the years. One of the biggest dangers in the immediate, large- 
scale use of teaching machines is that users will expect that no 
changes in supporting facilities and services are necessary or, 
still worse, that these devices can be expected to function rela- 
tively autonomously in the training environment. Although we 
know that this is not true, we are not yet able to specify clearly 
and with confidence the nature and amount of support that will be 
required. High priority should be given to studies which will 
assess the impact of automated devices on current training con- 
cepts and programs. 

c. Integration with other types of instruction. It is inconceiv- 
able that automated training techniques will ever completely re- 
place other forms of instruction. Rather, they will serve as sup- 
plements, in some cases rather substantial supplements, to so- 
called conventional methods. Considerable research will have to 
be accomplished before we can specify the ultimate role of these 
devices in training, but much can and should be done at the pres- 
ent time to assure their efficient and proper integration into 
current and planned training programs. 

d. Costing analysis. Machines cost money; sometimes substan- 
tial amounts of money. They also require maintenance; some- 
times substantial amounts of maintenance. All this adds up to a 
definite price tag, including both direct and indirect costs, attach- 
ing to any automated training program. People cost money, also, 
even though their cost is not always as obvious as with machines. 
Any decisions to substitute machines for people in a system will 
have to be based not only upon the relative capabilities of the two 
to accomplish the functions assigned but also upon the compara- 
tive costs involved. This will be especially true in the area of 
training where the military is quite cost conscious. Therefore, 
military psychologists will do well in their studies, if they not 
only gather data on training effectiveness but also make provi- 
sions for a thorough cost analysis of the technique being studied. 
Such cost figures undoubtedly will be required before the mili- 
tary will make any large-scale use of automated training tech- 
niques. 

e. Improved specification of training goals. A clear specifica- 
tion of the goals of the training is a requirement for developing 
any program of training or education; it is by no means unique to 
a program involving teaching machines. With teaching machines, 
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however, this issue is brought into clear focus because materials 
must be prepared down to the finest detail, tried out, revised and 
standardized before training begins. With teaching machines there 
is no such thing as going to class unprepared and "bluffing your 
way through." Nor is there much chance for hiding one's uncer- 
tainty about training objectives in wordy, highly generalized lec- 
tures. Rather, the exact material to be presented and mastered 
must be selected and programed in advance. This means that the 
teaching machine programer must come to grips with the goals 
of his training program early and without equivocation. The suc- 
cess with which this can be done is one factor which will deter- 
mine both the effectiveness of teaching machines and their accep- 
tance by training personnel. Therefore, a very important re- 
search requirement for the military psychologist is that of de- 
veloping new and improved techniques for the detailed specifica- 
tion of training goals and standards before training begins, rather 
than by an after -the -fact analysis of the graduates of training pro- 
grams. Such developments will have wide applicability to the field 
of training in general, but they are especially vital for the effec- 
tive use of automated training in the military. 

It is clear that there is no reason for the military psychologist 
with an interest in automated training to be idle in the next few 
years. There is much to be done, the time is ripe and the poten- 
tial payoff is high. No one can say for sure at present what the 
future holds for teaching machines. Several highly ingenious and 
imaginative accounts of teaching machines' possibilities have been 
put forward, e.g., Ramo (1957, 1959). We certainly do not deny a 
bright future for these devices, but we believe that any such bright 
future is best assured by a well conceived, step -by-step building 
of the present. About the only prediction we can make with confi- 
dence is that the next few years in this field will be active ones. 

There is a great deal of current activity and study in the area 
of teaching machines; these devices appear to have considerable 
potential for the field of military training, but there still is much 
work to be done before this potential can be fully realized. Actu- 
ally, today the practical use of self-instructional devices and ma- 
terials is merely in its infancy, but perhaps no single idea in 
military psychology since the selection test has created such in- 
terest and has had such far reaching implications for improving 
military operations. This is truly not only a great opportunity for 
military psychology but also a great challenge and a great respon- 
sibility. There is the opportunity to develop a tool which will 
bring considerable attention and great credit to our profession; 
there is the challenge of a psychological problem with depth and 
meaning; there is the responsibility to perform a significant ser- 
vice for the military organizations of which we are a part and 
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from which we derive our support. It will be important for mili- 
tary psychology to meet this opportunity, challenge and responsi- 
bility with enthusiasm and maturity. 
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SECTION 4. Computer-Based Teaching 

Systems 



Teaching machines usually are designed to produce results 
that approximate as closely as possible the results of studying, 
not with the average teacher in a classroom situation, but with 
an exceptionally skilled, highly trained, human tutor. In some 
ways the designers of machines and programs have been quite 
successful, but there are many problems still to be solved. One 
of these problems concerns the ability of a tutor to vary the kind 
of material to be worked on and the techniques to be used accord- 
ing to the progress the student is making at various stages in the 
learning situation. The fact that students using programed in- 
struction do make specific responses to a program makes it 
possible to construct a teaching system that can use and base de- 
cisions on this data. Because a great many decisions are in- 
volved, the electronic computer comes easily to mind as an ap- 
propriate instrument with which to implement such a teaching 
system. 

The possibility of actually constructing machines to meet the 
specifications described above seems to many to be as incredible 
as science fiction, yet actual attempts are being made in this di- 
rection. The first article in this section, for example, describes 
some of the theoretical bases on which the Systems Development 
Corporation has constructed a computer-based teaching system. 
A much less complex means of letting the students' responses 
control the organization of the program is described by Norman 
Crowder in the second article in this section. This article de- 
scribes systems that, although student-controlled, are much more 
economical than the usual high-speed computer-based teaching 
system. 

In contrast to these, the third article in this section is a theo- 
retical paper. It describes a model on which teaching-machine 
development and research can be based. Although the reader 
might have difficulty with the theoretical bases for the model, or 
may not understand its derivation, the discussions in the non- 
mathematical parts of the text are well worth the attention of any- 
one engaged in research and development of automated teaching 
systems. 
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At first thought, it might seem difficult to justify the use of a 
computer, with its associated cost and complexity, in an auto- 
mated teaching system. In this connection, certain statements 
should be made at the start. These are as follows: 

a. The finest automated teaching system can be no better than 
the instructional materials, or "items," used in the lessons. It 
is impossible to over-estimate the importance of well written 
items that are carefully prepared and directed toward clearly de- 
fined educational or training goals. 

b. Given a set of well prepared instructional materials, effec- 
tive automated instruction can be provided under many conditions 
with extremely simple devices or even with specially organized 
textbooks. 

c. For some training situations, a computer-based system 
would be unnecessary, uneconomical, and undesirable. 

THE ADVANTAGES 

On the positive side of the ledger, a computer -controlled in- 
structional system offers a number of valuable capabilities that 
cannot be matched by simpler devices or systems. Among these 
are the following: 

a. The operational characteristics of the modern digital com- 
puter are not fixed by the design of the electro-mechanical com- 
ponents. Rather, they are largely under the control of the "pro- 
gram"** that has been read into its memory. Thus, the com- 
puter-controlled instructional system provides great versatility 
of operation without the need for costly hardware modifications. 
This versatility is of particular value during the early develop- 
mental stages in the introduction of an automated instructional 
system. By appropriate program changes, the computer can be 
made to function in many different modes that can be compared 



*System Development Corp. (SDC) is a nonprofit corporation concerned 
with national defense and public welfare. It does not manufacture equip- 
ment for sale nor endorse individual manufacturers or their products. 

**Ih this article, the term "program" refers to a set of computer in- 
structions, not to the instructional materials seen by the student. 
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for relative training effectiveness in the particular instructional 
context in which the system is to be applied. In this way, it is 
possible to simulate and evaluate proposed, as well as existing, 
instructional configurations. 

b. With a computer-controlled instructional system, each stu- 
dent or trainee can be given the sequence of instructional ma- 
terials best suited to his needs. Highly refined techniques for the 
evaluation of student capabilities and performance can be built 
into the control program that determines the sequence of ma- 
terials and mode of instruction for each student. In this way, the 
student with more experience or higher aptitude can be taken 
through the entire course with a minimum of delay, whereas the 
slower, or less experienced, student can receive as much supple- 
mentary training as is necessary to bring him to the required 
level of performance. 

c. A general -purpose digital computer has capabilities that 
permit its use, not only for instruction, but for numerous other 
tasks that might be of importance to a military, educational, or 
industrial organization. Its extensive memory can be used for 
storage of student attendance and performance records as well 
as information about each individual's aptitude, family back- 
ground, previous training, or any other pertinent data. 

d. The computer's ability to perform rapid calculations per- 
mits the analysis of data for summary or predictive purposes, 
and its logical circuitry can be used to generate class-assign- 
ment lists and training schedules. 

e. By appropriate time -sharing techniques, a computer can be 
cycled through a number of tasks in such a way that it appears to 
be performing all these tasks simultaneously. In other situations, 
the computer might be used during certain hours of the day for 
instruction and during other hours for data storage and analysis. 

Computer-based teaching machines appear to have particular 
advantages for the types of training problems encountered in the 
military and in industry. Trainees in these areas vary markedly, 
not only in basic ability and personality, but in their educational 
and training experience as well. 

Research with the teaching system using a Bendix G-15 com- 
puter indicates that, with appropriate individualization of instruc- 
tion, students of relatively low ability and experience can be 
brought to the same level of performance expected of more intel- 
ligent and experienced students (Coulson, et aL, 1961). Thus, by 
adjusting the training to the individual differences of the trainees, 
the potential supply of adequately trained manpower can be in- 
creased. 

The time required for training is of critical importance in 
the military and in industry, especially where crash programs 
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in the interest of national defense call for rapid training and re- 
training. Research has shown that instruction time can be re- 
duced significantly by the branching method as compared with the 
linear method. (Coulson and Silberman, 1960). 

Training requirements in the military and in industry also call 
for diverse types of training. For example, some skills demand 
proficiency in perceptual motor performance; other jobs require 
both conceptual and motor learning (as with electronic trouble 
shooting); and depend, primarily, on verbal learning. The com- 
puter-based teaching system allows wide versatility in the de- 
sign of training procedures. This versatility should reduce the 
requirement for procuring many different types of equipment to 
fill various training needs. 

SOME POTENTIALS 

The capability of military and industrial organizations to ful- 
fill their objectives in the time limits set by contractual commit- 
ments and project deadlines is directly related to the efficiency 
of the functions and interactions among management, personnel, 
and training units. Computer-based instructional systems have 
decided potential for strengthening the power of these functions 
and interactions. 

The computer-based system could be used, not only for anal- 
yzing the responses of trainees and controlling the presentation 
of stimulus materials, but also for collecting and storing large 
amounts of information about the learning behavior of the trainees. 
This information could be used advantageously by both manage- 
ment and personnel. 

Timely information on the individual progress of trainees in 
training could be transmitted and displayed to management. In 
this way, management's ability to predict more accurately the 
availability of manpower for assignment would be increased. 

The detailed descriptive data of the trainees' learning behavior 
also could be used by personnel. Such information could be re- 
lated to vocational-aptitude and mental-ability test scores to al- 
low more accurate prediction of the individual difference vari- 
ables associated with rapid learning. Thus, the efficiency of se- 
lection of men for training could be increased. 

Recent developments and trends in both computer technology 
and information processing have additional implications for the 
application of computer-based instructional systems to the mili- 
tary and to industry. In the future, a computer-based library 
might be connected to a national repository of technical and scien- 
tific information. Trainees at consoles in this library could call 
up information on the current state of knowledge in any area. In 
a given study area, students or trainees in individual cubicles 
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might instruct themselves with the aid of a machine tutor. Each 
tutor could be linked, through the instructional computer, to a 
national information-retrieval system that would process written 
data into courses of study, develop diagnostic questions, and 
search for answers to questions asked by students. 

ITS FEASIBILITY 

Considering the cost of computers, how feasible is a computer- 
based instructional system? Initially, the cost might be prohibi- 
tive, but, in industrial and military organizations where compu- 
ters are now being used for various data-processing and control 
functions, the cost might be reduced by time-sharing existing 
equipment. Also, the research and development of small special- 
purpose computers in the future might reduce the cost substanti- 
ally. 

In the final analysis, cost must be considered in relation to 
system objectives. It seems unlikely that computer-based in- 
structional systems will make instruction less expensive, but 
they should certainly make it much more effective. 

TWO COMPUTER-BASED SYSTEMS 

The following paragraphs of this article describe work being 
done by System Development Corporation (SDC) on computer- 
based instructional systems. These systems, designed largely 
for research and development purposes, are not presumed to be 
ultimate or ideal configurations. A consideration of their func- 
tional characteristics, however, might give some indication of the 
potentials of the modern computer for training and educational 
systems of the future. 

An Experimental Facility for Individual Instruction 

Early in 1960, SDC completed work on a computer-based fa- 
cility designed for research in the field of automated instruction. 
This facility (see Fig. 4-1-1), which teaches one student at a 
time, consists of three major units: a Bendix G-15 computer, a 
random-access slide projector, and an electric typewriter. 

Bendix G-15 Computer. This computer serves as the central 
control unit for the system. It contains the master control pro- 
gram that governs the operation of the entire system. The com- 
puter determines, at all times during a training session, what 
materials are to be presented to the student. It analyzes student 
responses to the instructional material and compares these re- 
sponses with stored data. 

Rolls of paper tape, each of which can hold the necessary in- 
formation for a different instructional lesson, are contained in 
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Fig. 4-1-1 

easily interchangeable magazines. A bell mounted in the compu- 
ter frame can be rung by the computer for programed auditory 
signals. 

Random -Access Slide Projector. Developed by the engineering 
staff at SDC, this projector holds up to six hundred 35 mm slides 
in 15 magazines of 40 slides each. It receives instructions from 
the G-15 for selecting and projecting instructional slides in the 
sequence indicated by the computer. 

Electric Typewriter. Two primary functions are served by the 
typewriter, which is linked to the Bendix computer. First, the 
student uses the keyboard to insert his answers to the instruc- 
tional materials, which are normally in multiple -choice form. 
Second, the computer program takes control of the typewriter to 
print messages telling the student how successfully he has an- 
swered the questions. 

Functional Characteristics of the System. The student begins 
with a basic series of items, each of which he attempts to answer 
in multiple -choice fashion by pressing a key on the typewriter 
keyboard. He receives appropriate knowledge of results after 
each attempt. Every item presented to him is registered within 
the computer's control unit as having relevance to one or more 
specific topics to be taught. The computer keeps a record of the 
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student's performance on each item and a cumulative record of 
his performance on each of the topics covered by the items. These 
performance measures can include the student's responses times 
as well as his error count. 

If the student's performance falls below a certain level for a 
particular topic, he is "branched" to a special set of items that 
provides remedial work on that topic. He may be taken through 
the entire remedial routine or only apart of it. When the student 
returns to the basic series from the remedial routine he contin- 
ues until he has finished this series or until his performance once 
more falls below certain programed limits. In this event, he is 
again "branched," temporarily, from the basic series. 

In certain cases, a single missed item on a particular topic 
might be sufficient to take the student to a remedial routine. The 
system not only recognizes that an item has been answered incor- 
rectly but, also, considers the specific incorrect alternative 
chosen. Certain answers might be indicative of a serious mis- 
understanding by the student, in which case the system would 
"branch" immediately to a remedial sequence. 

If a student does very poorly in his training and requires ex- 
cessive remedial work (according to programed criteria), the 
system can take him completely out of the original series and 
"branch" him into an alternate basic series. This alternate 
series would represent, not simply a remedial routine, but a 
major change in the training approach. The original series 
might, for instance, use the procedure of first giving general 
rules and then presenting examples to illustrate the rules; the al- 
ternative series might reverse the order by gradually building up 
to a general rule through a progression of illustrative examples. 

In the event that a student's performance on some topics of a 
basic series is sufficiently high, the teaching machine can skip 
certain items of these topics, thereby allowing the brighter stu- 
dent to progress rapidly to more advanced topics. 

At various points during the training session, the system might 
require the student to indicate his degree of confidence concern- 
ing his own learning progress. If he expresses a feeling of con- 
fusion or lack of understanding, he might be taken to a remedial 
branch even though his actual performance is above the normal- 
criterion level for such a branch. An expression of great self- 
confidence, on the other hand, might cause the student to be re- 
turned more rapidly from a remedial routine to the main sequence. 

A Facility for Individual or Group Instruction 

The previously described computer-based system instructs 
only one student at a time. A newer facility (see Fig. 4-1-2), des- 



112 



COMPUTER-BASED TEACHING SYSTEMS 




Fig. 4-1-2 

ignated CLASS (for computer-based laboratory for automated 
school systems), makes the same individualized form of auto- 
instruction available to 20 trainees studying simultaneously. 
With CLASS, self -instructional programs can be combined with 
conventional instruction or with large-group automated instruc- 
tion that uses some of the properties identified with individual 
teaching machines (e.g., immediate feedback, controlled sequenc- 
ing, and prior programing). Through central mediation by a 
Philco 2000 computer, CLASS provides automatic data processing 
for many of the functions normally associated with educational 
systems, (e.g., scheduling, testing data analysis, and accounting 
control). 

This computer-based laboratory is part of the larger Systems 
Simulation Research Laboratory at SDC. It was designed as a 
general purpose facility in which existing or proposed educational 
systems can be investigated (Bushnell and Cogswell, 1961). The 
physical plant consists of an administrative area, a counseling 
and observation deck, a large automated classroom area, and a 
central projection room (not shown in the illustration). The class- 
room area is shown divided into two rooms by a sound-attenuating 
folding wall. This wall makes it feasible for instruction to be 
given simultaneously in two different subject areas and in two 
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modes of automated instruction, i.e., the group and individual 
mode. When the wall is folded away, a large group area is avail- 
able. 

For training directors or administrators., a 900-line-per- 
minute, high-speed printer under computer control makes avail- 
able information on the state of the academic progress of each 
student as well as other data useful for management purposes. 
Less urgent reports necessary for scheduling and curriculum 
planning and for budgeting and accounting can be retrieved when 
the computer is not selecting and analyzing instructional programs. 

The observation area includes a small counseling room in 
which is a teletype unit that is connected to the computer for re- 
cording and reporting diagnostic data and for calling up informa- 
tion needed by the counselor or interviewer. All of this data is 
stored on magnetic tape along with the student's educational re- 
cord (continually updated by his daily performance), previous 
school record, aptitude measures, and other relevant information. 

In the observation area, observers are able to monitor visually 
the behavior of students and teachers in the classroom and the 
counseling room through a one -way -vision glass window. In the 
event that the folding wall is used, as shown in the illustration, 
to divide the large area into two classrooms, a closed-circuit 
TV camera and monitoring unit permits observation of the remote 
classroom. Microphones placed in the classroom also permit ob- 
servers to monitor aurally the behavior of students and teachers. 

Individual Instruction with CLASS. Individualized automated 
instruction is made possible through the electronic components 
on each student's desk. Each student has the film viewer and re- 
sponse device shown in Figs. 4-1-3 and 4-1-4. The response de- 
vice is connected to the computer through a buffer system. 

The trainee begins his lesson with a sequence of educational 
items or frames presented by his film viewer. Some of the items 
are multiple -choice questions testing the student's knowledge of 
the subject material. The. student responds to the question by 
pressing one of five translucent keys on his response device, 
which key then lights up. After the student is certain of his se- 
lection, he presses the "Enter" bar, which transmits his choice 
to computer. The computer analyzes the response and provides 
knowledge of the results by turning on a light to the left of the 
response key that corresponds to the correct multiple-choice 
option. A red or a green light at the top of the response device 
also can be turned on by the computer to indicate the correctness 
of the choice. 

A cumulative record of the student's performance on each of 
the topics is kept by the computer. These performance measures 
include the student's response time, error count, and a record of 
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Fig. 4-1-3 





Fig. 4-1-4 
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specific alternate choices. As with the previously described sys- 
tem, if a student's performance falls below a certain level for a 
particular topic, the student is "branched" or detour ed to a 
special set of remedial items on that topic. The brighter student 
might not be remedially branched at all. 

When the student has read a frame of material, responded to 
the diagnostic question, and received knowledge of whether his 
response was right or wrong, he is ready to continue. He now 
presses the ' 'Enter " bar, which turns off the lights on his re- 
sponse device, and the number of the next instructional frame 
appears in the read-out panel. The student manually winds a 
film strip to that frame and repeats the cycle. 

At any stage during the training period, the student is allowed 
to review at his own option. If he presses one of the three buttons 
on the side of his response device, he will be instructed to turn 
to an item listing several areas for review that are related to the 
concept in which he has just been working. The appropriate re- 
view items are then read. 

If the student needs excessive remedial work, an auxiliary 
program can be read in from magnetic tape. The student now can 
proceed on a new series of educational items that could range 
from a different training approach to a change in media. 

The manually -ope rated film viewer that presents the instruc- 
tional items holds 35 mm film strip with a capacity for 2,000 
frames. The illumination of the screen is sufficient for reading 
under daytime ambient-light conditions. Each student's desk is 
equipped with movable independent-study panels that can be easily 
brought into place and then lowered to become a part of the desk. 

The Teacher's Components. The teacher has four sources of 
information available with which to monitor student learning be- 
havior. These are a teacher's display console, a film viewer for 
monitoring the educational program, a response device similar 
to the student's unit, and a digital display monitor. As can be seen 
in Fig. 4-1-5, the teacher's display console has ten "switchlight" 
buttons, each button corresponding to a single student's station. 
Five other buttons permit different kinds of actions to be taken by 
the teacher. An "Enter" bar allows the teacher to change her 
action before inserting it into the computer. 

When an instructor initiates a period of automated individual 
instruction, he inserts the student's assignment from punched 
cards and presses one of the "Action" buttons. This causes a 
student-history tape to be brought into the computer memory. 
The tape indicates where the student is in his lessons, his IQ, and 
his previous grade record. Another magnetic tape (the subject- 
matter tape) then is read into the computer to present the day's 
lessons to each student. 
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Fig. 4-1-5 

As the students enter their responses to the instructional items, 
the computer monitors individual student response behavior. 
When a student is experiencing difficulty, i.e., he makes a pre- 
specified number of errors, a significant error, or has an exces- 
sive response latency, a button lights up on the teacher's console. 
(This same light can be energized by the student who wishes as- 
sistance.) The teacher responds by pressing the button (which 
turns out the light) and by pressing an "action" button on his 
console. 

In response to this action, the computer furnishes a display of 
the student's performance through the digital display monitor. 
The student's performance read-out will contain the following 
information: 

a. The student's number. 

b. The reason for the alarm. 

c. The name of the concept or topic on which student is 
working. 

d. The number of errors on the present concept and the total 
number of items on this concept to which the student has re- 
sponded. 
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e. The number of errors for the lesson and the total number of 
items to which the student has responded. 

A second "action" button puts the teacher's response device in 
parallel with the response device of any student. The teacher 
then can monitor the student by observing the student's responses 
on the teacher's response device and by reading those same items 
on the teacher's film viewer with which the student is apparently 
encountering some difficulty. Other "action" buttons on the 
teacher's console allow him to start and stop lessons, reassign 
students to new stations, and call-up other analyses of student 
learning behavior. 

Group Instruction with CLASS. For group instruction, the TV 
screen and speaker serve as a common group output for different 
kinds of audio-visual presentations. An audio-visual system of 
remotely controlled slide and film projectors enables the teacher, 
from the front of the classroom, to call up films or other dis- 
plays; to stop the frame of the display, whenever it is appropriate, 
for group discussion; to repeat short film strips or sequences of 
film when needed; and to teach the group by an automated program 
that is sequenced and selected by the computer. 

During the actual class period, the instructor can schedule a 
film or series of slides for a specific time. When it is appropriate, 
the teacher switches on the TV screen and turns to the channel 
on which appears the first frame of the film or slide to be used. 
If, during a lecture, the teacher wishes to run a film or slide or 
stop on a frame, he can press the button appropriate to the action. 
With a call to the projectionist in the audio-visual center, he can 
arrange for additional films or magazines of slides. 

If the teacher wishes to intersperse his lecture with diagnostic 
questions to which the student should respond with his response 
device, the teacher can receive an immediate read-out on his 
digital display of certain group, subgroup, or individual student 
measures that indicate the degree of student understanding to 
help him to redirect his instruction. The teacher also can signal 
the correct response on the student's response devices by acti- 
vating the device at his desk. 

An automated group -instruction session, without the teacher, 
can be effected by computer control of the random-access pro- 
jector now being used in the experimental facility previously de- 
scribed. Group knowledge of results can be acquired simply by 
giving the correct answer on a slide following each item that has 
a diagnostic question associated with it. Individual feedback is 
supplied on the student response device. 

Further Development. Although the present configuration of 
CLASS is based on experience gained with the previously de- 
scribed computer-based teaching machine and on SDC's general 
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experience in the systems training and data-processing fields, it 
is not considered an ideal or fixed design. The results of re- 
search with the facility will undoubtedly lead to modifications 
and improvements. Moreover, developments in hardware and 
computer technology can be expected to cause continual re-evalu- 
ation of the design of CLASS. 

A prototype unit, manufactured by the Kalart Company, simul- 
taneously presents audio material over a headset and visual ma- 
terial on film strip. The sound record is photographically printed 
on the film. Eighteen seconds of sound recording can be stored 
on each frame. With the Kalart unit, a single student can be 
machine -tutored by a teaching machine that has both random - 
access slide and audio display. This could be an important capa- 
bility in comparing different sense modalities with students of 
different aptitudes for different subjects. 

A Simulation Laboratory. The use of a simulation facility like 
CLASS can aid substantially in the design and implementation of 
advanced instructional systems. By studying alternative methods 
and procedures in a laboratory where the environment is realis- 
tically simulated, many potential problems can be uncovered and 
effective operational procedures can be developed and tested. In 
addition to providing a simulation facility for research and de- 
velopment of instructional systems, a laboratory like CLASS per- 
mits training through simulation. 

The use of computer-based systems for such a purpose is not 
without precedent in the military and industrial environment. An 
IBM 650 has been used by the American Management Association 
for training top management through simulation procedures. 
Trainees make judgments concerning competitive business situa- 
tions and enter their decisions into the computer. The computer 
processes the information, interrogates the trainee further, and 
supplies him with knowledge of results indicating probable con- 
sequences of his actions (Craft and Seward, 1959). Within the 
military, the SAGE system training program provides simulation 
of inputs to the SAGE air -defense system through a digital com- 
puter. During an exercise each crewman in the system receives 
information that reflects the consequences of his own actions as 
well as the actions of other members of the system (Miller, 1960), 
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Simple Ways to Use the Student Response 

for Program Control 



By NORMAN A. CROWDER 
U.S. Industries, Incorporated 



When we consider the application of computing techniques to 
teaching machines, we come at once to some very fundamental 
issues in the field of automated instruction. As a beginning, we 
should distinguish clearly the structural differences between 
linear, or what I will call nonresponsive programing techniques, 
and branching, or what I will call responsive programing tech- 
niques. We must also examine the theoretical basis for these dif- 
ferences in structure. 

It is characteristic of all teaching -machine methods that they 
require, or at least provide an opportunity for, an active response 
on the part of the student at frequent intervals throughout the 
program of instruction. There have been two schools of thought 
as to the primary reason for eliciting the student's response. In- 
deed, the two schools represent historically independent develop- 
ments and spring from differing theoretical backgrounds. 

One school of thought, having its roots in classical experimen- 
tal psychology, views the student's response as an integral part 
of the learning process and, therefore, as a legitimate end in it- 
self. The adherents of this school are satisfied, in general, to 
make no further direct use of the student's response other than 
for its assumed effect on the student. This school has always, of 
course, emphasized the use of the records of student responses 
in the development and revision of a program, but not in the ad- 
ministration of it. Because the student's response is not used to 
control the program, the resulting programs are of the linear, 
or nonresponsive, type. 

The other school, having its roots in differential psychology, 
is primarily interested in utilizing the student's responses to 
control the course of the programed material presented to that 
particular student. Adherents of this second school generally 
have preferred to beg the question of exactly how the learning 
was to take place so as to focus attention on being able to deter- 
mine, from step to step in the program, whether learning had 
taken place. Given this information in usable form, it is then 
possible to arrange that the program be automatically modified 
until the desired result, for that student at that time, is attained. 
This second school, therefore, is interested in the student's re- 
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sponse as the primary datum required to operate a branching, or 
responsive, program. 

I would not argue that we should ignore the effect of the stu- 
dent's response on the student any more than we should ignore its 
possible use in automatically controlling and modifying the in- 
structional program. Nevertheless, it should be pointed out that, 
when we consider the use of computing apparatus to modify the 
instructional program for an individual student, we are interested 
primarily in this second use of the student's response and, by def- 
inition, are contemplating branching, or responsive, programs. 
To make full use of whatever computing capability we employ, 
we need to realize that a branching program has definitely differ- 
ent structural characteristics than a linear program and, also, 
has different capabilities. 

I hope I have not been tedious in drawing attention to this fairly 
obvious point that computer control implies branching programs. 
One reason I feel this point deserves some emphasis is that it 
was not very long ago that I heard a prominent educational psychol- 
ogist refer to those of us who were interested in branching pro- 
grams as a "vociferious minority" and suggest that our numbers 
should not be judged directly by the amount of noise we made. I 
take it, by the current interest in means of controlling branching 
programs, that the ranks are swelled somewhat, although per- 
haps with quieter members. This, however, is beside the point. 

There are two major aspects of branching, or responsive, pro- 
graming that I wish to call attention to before discussing my 
major topic, which is the choice of techniques within the general 
area of responsive programing. The first point I want to make 
is that branching, or responsive, programs will, I predict, utilize 
multiple choice rather than completion questions, and for a purely 
practical reason. It is simple, easy, and convenient to let the 
teaching machine, whatever form it takes, know which of several 
alternatives a student has chosen and to cause it to take appro- 
priate action on the basis of this choice. It is virtually impossi- 
ble, in the present state of the art, to let a teaching machine 
"know" what answer a student has given when the student has 
written out or "constructed" his answer. This purely practical 
consideration was, of course, my reason for originally preferring 
the multiple -choice -response mode, and, if I understand the re- 
search of the last year correctly, I believe that the theoretical 
reasons for preferring the constructed-response mode have been 
swept away by the experimental evidence. To use, I believe, 
Evan's phrase, multiple -choice responses are now allowed to sit 
at the same lunch counters as constructed responses. Reversing 
the usual geographic pattern, this form of integration seems now 
to have spread as far north as Boston and environs. 
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There are some curious cases where there is no difference be- 
tween the multiple -choice and the constructed-response format. 
These are those cases in which the universe of sensible responses 
is strictly limited to a finite number. Thus, there are only 26 
possible alternatives for the next letter in a spelling problem, 
ten alternatives for the next digit in a problem in decimal arith- 
metic, and only two in problems in binary arithmetic. In these 
cases, the distinction between the multiple-choice and con- 
structed-response format disappears. 

The second major point I want to make about branching, or re- 
sponsive, programs is that when we focus attention on the stu- 
dent's response as the primary datum needed to operate our 
branching program, rather than as a part of the learning process 
as such, we become aware that the questions in our program may 
serve a variety of different functions and that these different func- 
tions require different types of questions. A routine question on 
a routine step in the program should serve to do the following: 

a. Determine whether the student has learned the material just 
presented. 

b. Select appropriate corrective material if the student has not 

learned. 

c. Provide desirable practice with the concept involved. 

d". Serve to keep the student actively working at the material. 

e! Presumably, if the student gets the question right, serve a 
desirable motivational purpose. 

It is quite possible, however, that we will, in writing branching 
programs, write questions that serve none of these purposes. 
We may want a very "stiff" question, or a short series of "stiff" 
questions, to determine whether a student should skip a whole 
block of material. Such a question, as presented to the student, 
might not accomplish any of the purposes served by the question 
used in a routine program step. Again, this point might seem ob- 
vious, but I merely wish to point out that there is no reason to 
view each program step and the associated question as having the 
same function and, therefore, the same structural and statistical 
properties. For example, I think it desirable that, on a routine 
program step (if there is such a thing), no more than 15% of the 
students should select a wrong answer. But a major program 
branch might have a question that would be failed by 90% of the 
students. 

It might be somewhat less obvious that the alternatives pro- 
vided in a single multiple -choice question also might serve dif- 
ferent purposes and have different consequences. One alternative 
might be provided to catch a particular procedural error and 
lead to a single corrective presentation; a second might be such 
that it appropriately leads to a correctional subsequence; a third 
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alternative might catch an error of interpretation on a point made 
previously and lead the student back to that point in the program 
to work his way up again. All of these kinds of contingencies are 
fairly easy to provide for. 

After this long preamble about the rationale of responsive pro- 
grams, I would like to turn now to my major topic, which is the 
several means of accomplishing branching programing. We 
can distinguish two basic types of branching programs. There is 
the type of branching program in which there is a one-to-one 
correspondence between the student's answer choice and the next 
material he sees. Thus, the student's answer choice might lead 
him to turn to a particular page in a "scr ambled" book or to 
press a particular button on a machine, at which point a particu- 
lar, predetermined presentation appears on the viewing screen 
of the machine. This type of responsive programing requires 
no intermediate computation between the student's answer choice 
and the decision as to what material he will see next. I call this 
type of responsive programing intrinsic programing. When, in 
addition to the student's response alone, we use other data and 
perform further computation to select the next material the stu- 
dent should see, we have what I would call extrinsic programing. 

I would like to inject an historical note at this point. When I 
first began to work in this field in 1955, I assumed that any type 
of auto-instructional program could usefully be classified accord- 
ing to the technique used to provide the branching and, therefore, 
coined the terms "intrinsic" and "extrinsic" programing. It 
did not occur to me that anyone would seriously propose an auto- 
instructional program that did not provide any means of automati- 
cally varying the program to accommodate the needs of different 
students. 

The intrinsic programing technique has the obvious advantage 
that no complex computing equipment is required for its imple- 
mentation. The simplest device utilizing intrinsically programed 
material is the "scrambled" book, and, with relatively simple 
devices (simple when compared to the complex computers now 
being proposed), quite complex kinds of branching programs can 
be used. 

In principle, it is true that, with the simplest form of intrin- 
sically programed material (the scrambled book), any branch- 
ing program that depends only on the history of the student's 
choices can be accommodated, as can be seen in Fig. 4-2-1. A 
three-step program is shown, with two choices at each step. 
Each of the students arriving at a different point (A through H) 
on the third step has a different history, and yet in no step have 
we considered any data beyond the student's present location and 
his immediate choice. 
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StepS 



Step 2 



Step 1 




Fig. 4-2-1 

It would appear from Fig. 4-2-1 that the number of pages or 
locations to be provided becomes very large if the program is 
more than a few steps long. In practice, however, this is not the 
case because most of the student's history probably is irrelevant 
for the choice of what to do with him at any given point in the pro- 
gram. If we carry this idea to its extreme, we get the kind of 
pattern shown ih Fig. 4-2-2, which is the simplest kind of branch- 
ing pattern. In this pattern, branches representing errors are 
"cut off," and the student, after correction, is returned to the 
point at which he branched, as implied by the double arrows on 
the branches. With this type of sequence, the number of pages 
becomes manageable. 






Fig. 4-2-2 

A more complex, but still practically useful, sequence is shown 
in Fig. 4-2-3. The chief characteristics of the sequence shown in 
Fig. 4-2-3 is that a student making any error in Step A is placed 
in a subsequence that will lead him eventually to the next step, 
but only if he passes the criterion question at A ? , which covers 
essentially the same material as the question A. Again, notice 
that, in this sequence, the only data required to determine where 
the student should go next is his present location and his immedi- 
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Fig. 4-2-3 

ate choice. The branching program therefore, can, be accommo- 
dated in such a simple device as a scrambled book. 

A somewhat more complex device, the Auto Tutor Mark II 
shown in Fig. 4-2-4, handles similar material, but adds some 
new dimensions of control. The actual material is handled on a 
roll of 35 mm microfilm. The student sees this material projected 
on a viewing screen and has ten response buttons that he uses to 
answer the questions. The actual effect of each of the ten buttons 
is as follows: 

Button A advances film 1 step. 
Button B advances film 3 steps. 
Button C advances film 5 steps. 
Button D advances film 7 steps. 
Button E advances film 9 steps. 
Button F advances film 11 steps. 
Button G advances film 13 steps. 
Button H advances film 15 steps. 
Button I rewinds film 19 steps. 
Button R cancels previous excursion. 

Buttons A through H are the ones normally used by the student 
to answer multiple -choice questions. Each button advances the 
film the number of steps shown. If the student has chosen the 
correct answer to a question, the film frame on which he ends up 
will contain new information and a new question. When the student 
again presses one of the buttons to answer the new question, the 
film will now advance from the new location. If the student has 
chosen an incorrect answer, the material he sees will, in the 
simplest case, present correctional material and then direct him 
to press the R or return button. This action will reverse the ex- 
cursion the machine has taken immediately previously and put the 
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Fig. 4-2-4 

student back on the original question frame to have another try at 
the question. 

This is the sequence of events, somewhat more conveniently 
arranged, that would occur if the student were using a scrambled 
book. Having the material inside a machine under mechanical 
control, however, enables us to add some very interesting new 
control features to the material. These controls are brought about 
by a code on each film frame that determines which buttons on the 
machine will be allowed to operate when the student is viewing the 
frame. There are four codes on the Mark n AutoTutor with the 
following functions: 

a. Code X-all buttons work. This is the code used routinely 
on a right-answer frame. The student has just chosen the correct 
answer to the previous question. We are satisfied with his per- 
formance and have no reason to restrict the student's activity, 
nor does the student have any reason to do anything but what we 
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want him to do, which is to choose an answer to the new question. 
If the student wants to review the previous right-answer frame, 
he can do so by pressing button R, which we allow to work in this 
case. 

b. Code Z only the button R works. All other buttons do not 
work. This is the code normally used on a simple wrong-answer 
frame. The student on such a frame has made an error, and we 
want him to read the correctional material and then to return to 
the question page. We arrange the situation so that only button R 
works in this case. 

c. Code Y only buttons A through H work. This code is used 
on a wrong-answer frame that introduces a correctional subse- 
quence. The student has made an error, and we want him to work 
forward through a correctional subsequence. In particular, we do 
not want him to return to the question to try another answer. 
Therefore, we do not allow the button R to work from this frame. 

d. Code W only the I button works. The I button on the Mark 
II is the only button that runs the film backward. It, with the W 
code, is used to run the student back in the program, to review 
material previously covered, or, in a sequence of backward ex- 
cursions, to bring the student back to the beginning of a complete 
section of the program. 

With the use of these film codes, which appear on each frame 
of instructional material and are read by two photo-cells in the 
machine, it is possible to achieve great flexibility and, at the 
same time, exercise complete control over the options available 
to the student. It is literally impossible for the student to advance 
from frame 1 to, say, frame 400 of a program until he achieves 
the criterion performances we have required on the way. 

I suppose it is a nice point whether the Mark II AutoTutor, 
with the film codes described, uses intrinsic or extrinsic pro- 
graming. Whatever conclusion we might come to on this ques- 
tion of definition, it is nonetheless true that we use only data de- 
rived from the student's present position, the film frame he is 
presently viewing, and his immediate choice to determine what 
will happen to him next. My point in presenting this information 
in this much detail is to give some idea of the complexity of pro- 
graming that can be accommodated without involving the more 
complex types of computing apparatus. 

I would like to make one final point bearing on this matter of 
complex control of teaching machines. To my mind, it is the logic 
of the subject matter itself that always must be allowed first 
consideration in designing a program. No amount of computing 
capability can substitute for clear, well written programed ma- 
terial. This is unfortunate because it is easier to draw logic dia- 
grams for computer programs than it is to write effective pro- 
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gram material. Nonetheless, it is the material that will ultimately 
determine the success of the program. I believe that the field of 
programed instruction is now beginning to free itself of the re- 
strictions resulting from adherence to a grossly over-simplified 
learning paradigm; I would hate to see it immediately succumb 
to a second type of restriction imposed by the possibility of using 
very complex formal control at the expense of paying close atten- 
tion to the material to be taught. 

* * * 

This paper is based, in part, on a paper presented to the Confer- 
ence on Application of Digital Computers to Automated Instruc- 
tion, October 10-12, 1961, in Washington, D.C. The Conference 
was co-sponsored by the Office of Naval Research and System 
Development Corporation, 
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AUTHOR'S NOTE: Since we wrote this article in May, 1960, sev- 
eral teaching devices using digital computers have been devel- 
oped. Some have merely used the computer to make a selection 
of the items to be presented to the student based on the student's 
responses to previous items and a preconceived, fixed set of 
branching rules. Others have used the computer to gather and 
process data on student performance for periodic review by the 
experimenter or teacher. 

Both of these functions probably can be accomplished in afar 
less expensive fashion than by using an "on-line" computer in 
the teaching system. The promising future for computer appli- 
cation to teaching systems lies in the computer's capability of 
modifying the logic decisions governing the presentation of ma- 
terial to the student while he is learning. If the logic decisions 
are "canned" in fixed computer instructions (program), the speed 
capability of the device is not being used at its full potential. 

Computers are being used to regulate the presentation of items 
and to record and analyze student responses, but they are not yet 
being used as experimental tools to systematically vary or "per- 
turb" the learning situation to indicate fruitful directions for 
change to the experimenter or the machine itself. Currently, the 
time lapse between experiments is too long, and therefore oppor- 
tunities to improve the educational process are too widely spaced. 
In production terms, experimentation is conducted on a "batch" 
rather than a "continuous flow" procedure. 

There are so many factors or variables which affect the teach- 
ing situation that the current theories of learning, appealing in 
their simplicity and elegance, are probably inadequate because 
the simplicity is an oversimplification. 

Traditionally, whenever educators and behavioral scientists 
have encountered a complex problem containing many interacting 
variables, they have dissected the problem into a host of two or 
three variable problems. Each sub-problem then becomes the 
subject for separate investigations, research papers, and often, 
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contradictory reports. However, two things are almost certain: one, 
the original complex problem is lost sight of in the mass of sub- 
problem investigations; and two, the results of the sub-problem 
investigations can never be combined into one meaningful whole. 

The elements in an automated teaching system are no less 
complex than those of other teaching systems, and, because auto- 
mated teaching starts with the premise that teaching can be treat 
scientifically, there is certain to be a new rash of scientific ex- 
periments on the one hundred or more variables involved in auto- 
mated teaching systems. If these experiments are not to go on 
inconclusively and indefinitely, we must consider some way to 
simultaneously improve our teaching -learning techniques (which 
is really our primary goal) and at the same time experiment with 
the variables of interest so we can make further improvements. 
This article will try to suggest one way in which this double ob- 
jective may be accomplished. 

A previous article (Roe and Moor, 1960) indicated a method for 
analyzing curricula, and highlighted some of the inadequacies of 
the current method of grouping blocks of learning into semester 
courses. It suggested that a much more detailed breakdown of 
subject matter would be desirable, and that the major difficulty 
in making such a breakdown is the complexity involved in the 
proper ordering of this greater number of "knowledge bits". 

Proponents of automated teaching devices claim that mechanica. 
or electronic modes of presenting knowledge bits will permit stu- 
dents to respond more or less continuously to learning stimuli, 
to proceed at a pace consistent with their individual capabilities 
and based on their individual learning history. While there is no 
doubt that such devices can ensure the presentation of knowledge 
bits in a precisely ordered fashion for optimum individual learn- 
ing, they can only do so if a precise ordering has been pre-deter- 
mined and the task of ordering knowledge bits is still the most 
crucial and most difficult part of teaching automated or other- 
wise. 

The proper ordering of knowledge bits for individual learning 
can be broken down into the following three separate functions; 

a. Coding, or the statement of knowledge bits in a form that 
can be conveniently stored and retrieved. (This problem will be 
discussed in a subsequent article.) 

b. Storage and retrieval of pertinent information. 

c. The decision as to what sequence of information bits should 
be presented to each individual student. 

The human mind can perform all of these functions. However, 
the average human mind has certain limitations in the amount of 
subject matter information it can store and quickly retrieve from 
storage (memory and recall). Also involved in the storage and re- 
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trieval problem is the information of individual student and learn- 
ing factors on which to base the decision functions. When one con- 
siders the vast amount of continuously changing information, both 
related to subject matter and decision functions, one soon realizes 
that some external extension of the human mind is desirable. 
Such an extension of the human mind is found in modern electronic 
computer complexes. Indeed, such computer complexes can be 
broken down into the information storage and retrieval functions 
and decision functions. 

Let us consider some of the current capabilities and limitations 
of electronic computers. A computer has two types of "memory" 
for information storage: an internal memory of limited capacity 
but with millisecond speed of information retrieval, and an exter- 
nal memory of almost unlimited capacity (size of building and cost 
are the important limitations), but with slower speed of retrieval. 
The internal memory can be reserved for decision functions (in- 
cluding decisions on where to "search" for information in the ex- 
ternal memory) and the external memory reserved for storage of 
subject material and information on individual students and learn- 
ing factors. A complete college textbook can be stored on a por- 
tion of an external memory tape so that any section of the text 
can be retrieved in from milliseconds up to 3 min, depending upon 
the location of the desired information on the tape. This maximum 
time for retrieval from the external memory (student waiting time) 
is excessive for most direct machine teaching modes. Random 
access external memory units, currently in production, can re- 
duce the maximum retrieval time to one second, a wait period 
that would be hardly perceptible to the student. While the initial 
cost for storage of, say, 1,000 college textbooks would be high, the 
unit cost per student educated would drop below the current cost 
of textbooks in a short time. At this stage in the development of 
automated teaching programs, it is difficult to predict whether 
such programed material, if integrated into an overall educa- 
tional system, would require more or less storage space than 
standard textbooks. Whichever proves to be the case, it is appar- 
ent that a 1,000 -book tape library, any portion of which is available 
within seconds, far exceeds the storage and retrieval capabilities 
of the average instructor. 

Now let us consider the decision function of a computer. Elec- 
tronic computers evoke decisions on a "yes" or "no" (digital) 
basis. Higher level decisions are made by logical combinations 
of "yes-no" elements. Since the elemental "yes-no" choices 
are made in the internal memory in less than a millisecond, fairly 
sophisticated decisions, or logical combinations of "yes-no" can 
be made in a very small span of time. However, it is not difficult 
to conceive of decision functions which will exhaust both the inter- 
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nal storage capacity of any conceivable machine and/or the practi- 
cal time available for making the decisions. An example frequently 
quoted is that of writing out all the possible arrangements of 10 
persons in ten chairs (10!). This would require 3,688,800 items, 
a large number, but one still within the realm of computer capa- 
bilities. However, if the problem were printing out all the possi- 
ble arrangements of 20 persons in 20 chairs (20!) and it took the 
computer one millisecond to write out each name -position, it 
would take the computer 500,000 years to complete the task. 

Let us examine now the duties we might expect the computer to 
perform, and see how well these duties are consonant with fore- 
seeable machine capabilities. We would expect the machine to 
store vast quantities of information in a readily accessible fashion. 
It has already been indicated how this may well be accomplished 
more handily and probably more economically than by current 
methods. We would expect the computer to keep a record of the 
student performance. Our educational system already makes a 
sizeable investment both in physical plant, administrative and in- 
structors' time to keep a bare minimum of records on gross stu- 
dent performance. Here again, a computer could do a more de- 
tailed job and probably at lower overall cost. We would expect 
the computer to be able to accept (or reject) new information and 
to re -evaluate old information (correct and adapt). Eventually, 
we would want to be able to present random bits of knowledge to 
the computer and have the computer evaluate whether these bits 
should be added to storage, where they should be added, and when 
and to which students it would be best to present these new bits 
of information. This latter requirement staggers the imagination 
when one considers the large number of variables (about which 
we still know very little) which would influence the best choice. 
A few of these variables are the following: 

a. Student preknowledge. 

b. Student intelligence (past performance). 

c. Level of student aspiration. 

d. Learning environment. 

e. Attention factors. 

f. Size of learning increment. 

g. Kind of subject material, 
h. Reinforcement schedule, 
i. Feedback. 

j. Variability of learning schedule. 

k. Type of response. 

1. Automation apparatus. 

m. Learning period. 

In essence, we are asking the computer to become a continuous 
and vital link in the education process. We are asking it to sense 
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and respond to everchanging subject matter, student and evalua- 
tion criteria inputs. We are even asking that it sense and respond 
to new variables. These are tasks which have traditionally been 
performed directly by human minds in more or less intuitive 
fashion, primarily because of the limited ability of the human 
mind to deal quickly and precisely with vast amounts of data. 
When we ask a computer to make some of the decisions formerly 
done by man, someone must write the rules by which the computer 
will make these decisions. We could even write rules for making 
rules, so that the computer can use the higher order rules to make 
the lower order rules to use in making the decisions. But no 
matter how high up we go in computer control levels, ultimately 
we must insert a human " super -control instruction". The advan- 
tage of placing the human instruction on higher levels of control 
is that we can make accessible to the computer large quantities 
of data for making the decisions on the lower levels, and thus re- 
duce the magnitude (though not the "spirit") of intuitive decisions. 

How then can a computer handle this ever-shifting multi-vari- 
able problem? Computers can perform testing of hypotheses. 
However, such tests generally only indicate whether or not the 
sample central measure (mean, mode, or median) of one thing is 
significantly different from the sample central measure of some 
other thing. If another variable becomes interesting, a separate 
hypothesis is formed. In each case the variables are dealt with 
separately. Slightly more sophisticated techniques permit one to 
consider a number of variables at a time, but still, one only con- 
siders the significant likelihood of equality of selected parameters 
or existence of interactions. Though data can be gleaned about the 
relative magnitude of the significant variables, this is often ig- 
nored, particularly by behavioral scientists. This may be partly 
due to their difficulty in combining the variables of interest in a 
fruitful manner. Coming back to computers, if even a modest num- 
ber of readings on a limited number of variables is considered 
simultaneously, we exhaust the computer capabilities. If we con- 
sider only one variable at a time, we may overlook important inter- 
actions. Sometimes we take a large number of variables and find 
the (often approximate) functional relationships between these 
variables and one particular variable of interest, say, cost. In 
Fig. 4-3-1, if the ordinate "P" is performance or payoff, a single 
variable might be time, another, machine complexity. Both of 
these could perhaps be related to cost so that a composite func- 
tion could be graphed. If this composite function has a maximum, 
then differentiating the function will yield a simple solution for the 
optimum cost -performance point. However, it is suspected that a 
large percentage of problems, particularly in the behavioral 
sciences, do not exhibit such simple optimum points. Rather, they 
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may have a number of nodes or none at all. If no node exists, then 
we generally specify constraints or limits to the area of interest 
in the function space. Such a set of constraints is shown in the 
lower half of Fig. 4-3-1, where the upper limit might be the cost 
at which the system could no longer maintain itself (go -bankrupt) 
and so would not be around to enjoy the increased payoff. The 
lower limit might be the minimum cost necessary to achieve an 
acceptable level of payoff. Any point on the function between these 
constraints is a "satisfactory" solution in the absence of addi- 
tional evaluative criteria. And since this function is a composite 
of more than one variable, in the multi-variable space we may 
have a multi -dimensional surface of "satisfactory" solutions. 
There does not appear to be a single optimum solution for such 
problems. 

There is a technique developed by G.E.P. Box (Davis, 1956) for 
handling the kind of problem in which we are interested. Box con- 
siders some multi-variable response surface, as shown in Fig. 
4-3-2. This response surface is supposed to represent the "real" 
yield or payoff corresponding to each point on the variable coordi- 
nate plane. For the sake of illustration, Fig. 4-3-2 shows a two- 
variable response surface, though the method is equally as valid 
for systems with more than two variables. This relatively simple 
surface is shown here merely to introduce Box's technique. Later 
we will show how the response surface model can be modified to 
meet with the requirements of a computer -controlled teaching 
system. In the case shown in Fig. 4-3-2, we are interested in 
finding the maximum pay-off point, shown by a dot on the response 
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surface. However, initially, we have no idea where the surface 
is, nor where is the optimum point on the surface, and can have 
no idea until we sample from the "real world" at various x l x 2 
points and observe the payoffs at these points. Box suggests that 
only a sample reading at just a few x l x 2 points need be taken 
initially, in order to estimate the slope of the surface at some 
region in the general area of interest. The direction of maximum 
slope will be the direction in which to move before taking the next 
few sample points. That is, one takes the path of steepest ascent, 
Eventually, a region will be reached within which the slope is zero 
in all coordinate directions, and this will be the location of the 
maxima (or minima). 

Any surface of k variables can be approximated to whatever 
accuracy one wishes by using a polynomial equation including 
terms up to whatever degree as is felt necessary to give the de- 
sired accuracy. The general k -variable response surface would 
be represented by 



013*1*3 



Box suggests that, for the first test or experiment where one 
may be finding the path of steepest ascent, a first degree approxi- 
mation (or plane) to the response surface could be adequate to 
determine the direction of maximum slope. Therefore, use 

7?= + /^ + /3 2 # 2 

Higher degree terms are used when it is suspected that a first 
degree approximation is not adequate, and particularly when 
small slopes are encountered, such as in the region of the maxima. 

If we are primarily interested in finding an optimum point for 
our system operation, or if we wish to improve our system in the 
quickest and least expensive way possible, it would seem that a 
technique which points out the proper direction for movement or 
change is more desirable than one which exhaustively defines the 
nature of a specific region. This is especially true if the first 
region is not one of interest (not near an optimum point). 

In the usual analysis of variance technique, a large number of 
samples are taken at the sample points to reduce the error in 
estimating the location of the pay-off points on the "real" re- 
sponse surface. Box's technique takes only one reading at each 
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coordinate point. It might be argued that this does not give any 
estimate of the sampling error at these points. True, but what is 
the sense of carefully defining the response surface in a not-inter- 
esting region? Furthermore, the coordinate system we have 
chosen may have been a first approximation, showing only the 
variables which seemed at the moment to be the variables of pri- 
mary interest. The issue here is to consider the dynamic pro- 
cess; that is, the steps or moves necessary to improve the sys- 
tem, and not the extreme accuracy of estimating a given surface, 
particularly in a coordinate system which itself may be only a 
poor approximation. As one moves to regions of less change in 
slope, then one should consider greater accuracy of estimating 
the surface, and, also, should consider re -introducing some of the 
less important variables which may have been left out in the 
earlier investigations. 

The mathematical treatment is also attractive, since it allows 
us to make decisions (in the direction of movement) without the 
accumulation of a large number of sample readings. This is im- 
portant for two reasons: if the computations are performed by 
a computer, the method saves memory storage space; and, if the 
data are being accumulated while students are actually learning 
on our automated teaching system, then almost continuous modi- 
fication in the "best" direction can be made in the teaching pro- 
gram, and not only after the accumulation of vast amounts of 
sample data. When the teaching system can change and adapt it- 
self continuously in response to the different inputs, when it is 
able to increase the order in the learning-teaching situation at 
the expense of chaos, when it is shaped by, and in turn shapes its 
student (and social) environment, then we begin to have the mak- 
ings of a self -organizing system (Foerster, 1959). 

In Fig. 4-3-2, we have illustrated a response surface which 
did have a maxima. It has been previously suggested that many 
systems do not have maxima, and may be limited only by some 
time -dependent constraints. Figure 4-3-3 shows such a two- 
variable response surface. 

Usually there is some effort (or cost) involved in taking sample 
measurements so that, theoretically, we can imagine a specific 
amount of effort (or cost) for each point on the coordinate plane. 
The effort amounts for all points on the coordinate plane would 
describe a new response surface , where is of the same form 
as rj (see Fig, 4-3-3) 
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If, now, we are interested in 0, the net payoff, and can be ex- 
pressed as gross payoff minus effort, then 
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as shown in Fig. 4-3-4. It is interesting to observe that <>may 
have an optimum point, even though 77 and do not. 




The mathematics used in choosing a path of steepest ascent 
lends itself to incorporating other concepts. For example, utility 
theory (Chernoff and Moses, 1959) can be wedded to Box's path 
of steepest ascent by multiplying the slope of each variable (in a 
specific region) by a weighting function. In effect, what one is 
doing is changing the relative scales of the coordinate system. 
Another place where utility theory could be employed is in choos- 
ing the size of step to take after one has determined the direc- 
tion in which to move. Here the choice is between a very long 
step, which may overshoot the maximum point, versus a very 
small step which may not move us very quickly to the maximum 
region. 

If the weighting functions change with each sampling (or ex- 
periment) then is a function of time (or more properly, of the 
sequence of experiments). In other words, the response surface 
does not remain fixed, but changes as a result of the particular 
set of sample points or experiments we choose at each step. This 
again reminds us of one of the characteristics of a self -organizing 
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system; i.e., one which changes in response to the environment, 
and simultaneously reacts upon the environment. And this is what 
would happen in our hypothetical self -organizing teaching system. 
The students would be learning, the system would be evaluating 
what the students learned, and as a result, changing what the stu- 
dents will learn in the future, and, thus, the future evaluation of 
what students learn. 

If is the state of the response surface at the beginning of the 
story, where (j> is a function of x, /3 and y, or 



then after the first step the response surface ^ can be expressed 
as a function of cj> Q 
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which is a Markov chain. 

Such a Markov chain is susceptible to treatment by Bellman 
andKalaba's dynamic programing techniques (see references). For 
example, they define an optimal policy as having "the property that 
whatever the initial state and initial decision are, the remaining de- 
cisions must constitute an optimal policy with regard to the state 
resulting from the first decision". We can see that if Box's con- 
cept of choosing the path of steepest ascent is looked upon as a 
sequence of maximizing steps, then it is similar to dynamic pro- 
graming. It is hypothesized that the method of dynamic pro- 
graming is extensible to situations where the final state is un- 
known, though currently available examples have dealt with opti- 
mum decision functions, connecting a known initial state and a 
stipulated final state. 

One difference (possibly attributable to the different backgrounds 
of the originators) between dynamic programing and the method 
of steepest ascent is that, in the former, the emphasis is on mathe- 
matical expression of the functional relationships and solution of 
the problem by simulation; whereas, in the latter, the emphasis 
is on experimental sampling and solution in the experimental or 
"real world" situation. Nevertheless, both techniques offer inter- 
esting possibilities for use in a computerized automated teaching 
system. 

When searching for an optimum teaching system, one should not 
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only relate the system of interest to the neighboring or coincident 
systems, but one should also recognize the arbitrary nature of 
system boundaries and/or variables. Automated teaching systems 
can be conceived on too narrow or on too large a scale. If one 
conceives of some non-computerized system, then the tedium of 
ordering the learning bits, evaluating them, keeping records on 
student performance, and servicing a large number of (presum- 
ably small and individually inexpensive) machines could mitigate 
against their widespread use. If, on the other hand, one conceives 
of a system which will make precise evaluation of a host of vari- 
ables which maybe influencing the system, then the size, com- 
plexity, and cost of the computer element of such a system may 
be beyond the realm of practicality. 

Somewhere in between, a computerized system which uses ap- 
proximating techniques to determine the direction for change, 
should prove currently feasible. As long as the initial state of 
the system is sufficiently large to assure its survival, its self- 
organizing attributes should allow it to rapidly increase its per- . 
formance capabilities, and, therefore, to grow at the expense of 
other existing educational systems. A relatively small computer- 
ized system could start at any point in the learning sequence, 
and, by first focusing its attention on the analysis and ordering 
of knowledge items in that and adjacent areas, continuously grow 
(and continuously re -order its items in the light of its growth). 
The larger the system would become, the more powerful it would 
be to perform the ordering task, and the faster it could "eat" 
neighboring less -organized systems. Eventually, one could visua- 
lize an inter -connected network of learning centers, where the 
experience gained in one center would immediately be made 
available to the "memory" of all other centers. Instructors at 
any center could feed in discrete information bits at any time 
and in any order and these inputs would be automatically pro- 
cessed by the decision functions of the computer. The presenta- 
tion of information would be tailored to the pragmatically deter- 
mined needs of each individual student. 

Nothing has been said here about replacing the human contact 
of student with teacher and student with student. The system en- 
visioned above could exist with and without such contact. The 
choice depends on the social mores of the times. Today, the 
thought of having students working "face-to-face" with a teaching 
machine most of the school day is frowned upon (much as spend- 
ing a few hours of each day in an automobile was frowned upon 
fifty years ago). However, the very existence and use of a large 
scale automated teaching system will, in time, influence our social 
mores, and if one feels (and this is an item of faith) that the hu- 
man aspects of student-teacher and student- student relationships 
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should be preserved, then one should hedge the automated teach- 
ing system with carefully conceived constraints. 

This report has suggested that automated instruction can be 
initiated immediately, even though the nature of all the variables 
are not yet understood. The instruction process and the experi- 
mentation on that instruction process can (and almost must) pro- 
ceed simultaneously. 
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SECTION 5. Market Analysis 
and Economic Considerations 



This section is devoted to the economic considerations of pro- 
gramed instruction. Attention is directed to the probable pro- 
ducers and consumers of this material, projected annual sales, 
marketing approaches, preparation time, cost of individual les- 
sons, and production factors. Although exact cost figures cannot 
be given, the authors are able to estimate the range of values 
and to indicate the parameters determining higher or lower es- 
timates. 

Both the article by Kincaid and Sorenson and the Basic Sys- 
tems staff memo serve to describe the potential consumer. Kin- 
caid and Sorenson take the broad view and consider many differ- 
ent markets; these include primary and secondary schools, col- 
leges and universities (including graduate schools), vocational 
schools, correspondence schools, and industrial and military 
schools. Analysis of past and current educational budgets en- 
ables the authors to project future expenditures in each of these 
areas. 

Interest in the Basic Systems staff memo is focused on job 
training problems in American industry. A statistical overview 
indicates a growing disparity between present training capabili- 
ties and industrial needs for skilled personnel. Specific job 
training areas are analyzed and reveal the magnitude of the 
training problem, the inadequacy of conventional procedures, and 
the potentials of programed instruction. In addition, a detailed 
analysis of a representative industry, the electronics industry, 
indicates those job categories most amenable to programed in- 
struction. 

In his article, Wilbur Ross has analyzed the forthcoming in- 
dustrial market for programed instruction and has suggested a 
marketing strategy for programing agencies. Programed in- 
structional material will alleviate many industrial-training 
problems and, hence, will gain widespread acceptance, although 
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this does not necessarily presage success for the companies 
that prepare this material. In large corporations, internal tech- 
nical divisions will acquire some program -writing sophistica- 
tion and will give out contracts only for special-purpose pro- 
grams. In response to this contingency, programing agencies 
must develop acceptable marketing strategies. One possible 
approach involves the preparation of proprietary material of 
general interest to many consumers that can be inexpensively 
marketed for a large segment of industry. 

The economic factors in constructed-response program prep- 
aration and program reproduction are discussed in articles by 
Klaus and Lumsdaine, by Deutsch, and by Margulies. Klaus and 
Lumsdaine point out that program cost is derived from prepara- 
tion cost and reproduction cost. Of these, reproduction costs 
are greater if large numbers of users are anticipated. In this 
article, consideration is given to the cost of reusable and non- 
reusuable paper textbooks, the role of inexpensive devices, and 
the possibility of using microfilm. 

Dividing programs into difficult, medium, and easy, Deutsch 
provides estimates of the different time factors involved. He 
considers the man-hours required to produce programs, the 
time required by trainees to complete this material, and the 
expected reduction in training time that will result. These es- 
timates are analyzed in the light of modern industrial require- 
ment, 

The article by Margulies is directed to the training director 
of a company newly initiating courses using programed mate- 
rials. Particular consideration is given to the problem of which 
courses should be undertaken first, the economic factors in- 
volved, the difficulties of measuring success, and the problems 
of integrating a programed course into the total training system. 
An attempt also is made to provide cost estimates of material 
contracted to a programing -preparation company. 



What's Ahead in the Programed-instruction 

Market? 

By PHILIP H. SORENSEN and HARRY V. KINCAID 
Stanford Research Institute 



Our article deals with what might be called the logistics of 
an educational innovation in this instance the design, develop- 
ment, testing, production, marketing, and maintenance of pro- 
gramed instruction and teaching machines. We will consider 
present applications and prospects over the long run for the 
employment of programed-instruction procedures by four 
broad classes of users: the regular schools from kindergarten 
to graduate school; business, industry, and government; the 
military; and individual people. Consideration will be given 
to present applications and prospects according to the nature 
of the learner, the program, the device, and the subject matter. 

This article is not intended to be a market forecast in the 
traditional sense. It is, rather, a prediction of the impact of 
an innovation on an "industry" education- of fairly well known 
dimensions. Some facts about current uses will be reported. 
Lacking facts, we will attempt plausible predictions. Inevitably, 
the forecast will be speculative and limited in its accuracy. 

Before proceeding to a detailed discussion of the logistics of 
this innovation, we should re -emphasize some of the salient 
propositions appearing elsewhere in this book and in the litera- 
ture. Programed instruction makes good sense. Teaching 
machines work. These generalizations are supported by experi- 
ments in the laboratory and experiences in the classroom. 
Programed-instruction procedures employ widely accepted 
principles of learning such as the following: 

a. Learning motivated by success tends to be more effective 
than learning motivated by fear of failure. 

b. Active participation by a learner usually leads to faster 
and more effective learning than does passive reception of 
information. 

c. Materials and tasks that make sense to the learner are 
mastered more readily than nonsense materials or tasks that 
are not understood. 

d. Immediate information or "feedback" about the nature of 
a good performance knowledge of mistakes or successful 
results makes for more effective learning. 

Programed instruction, when properly devised, employs all 
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these principles. But to say that programed instruction is 
"good sense" and that teaching machines "work" does not im- 
ply that all specific issues are resolved. We still do not know 
(and it will be many years before we do know) precisely the con- 
ditions under which programed instruction works best -the kinds 
of learners for whom it works best, the subjects that are most 
amenable to programing, the conditions under which the 
cost of programing justifies "payoffs" in learner performance. 
There are a myriad of more subtle issues of research interest 
such as strategies of programing, methods of stimulus presenta- 
tion, kinds of responses to require, size of steps or frames, the 
utility of prompts and cues, and so on. 

The many important unanswered questions regarding pro- 
gramed instruction and teaching machines remind us that its 
limits are but approximately known. It is probable that the 
power of programed instruction (coupled with automated teach- 
ing devices as a research tool) is barely sensed, let alone fully 
appreciated. 

For the training specialist whose attention is snared by en- 
thusiastic forecasts, there is some danger of viewing programed 
instruction and teaching machines as an answer to problems for 
which they may be only incidentally applicable. But even uncrit- 
ical acceptance might be a blessing if it stimulates hard thinking 
about ways to resolve problems of personnel performance. The 
person who seriously considers applying the concepts of pro- 
gramed instruction to real problems of improving personnel 
performance is bound, ultimately, to come to grips with these 
problems in a manner that might be new to him. 

The application of programed instruction demands a critical, 
rigorous, and tough-minded appraisal of performance objectives 
in specific behavioral terms so as to permit measurement of 
the degree to which the objectives are achieved. As the analysis 
of objectives is undertaken, alternative routes to these object- 
ives might suggest themselves. Means that do not involve train- 
ingjob simplification, modifications in supervisory practices, 
or variations in personnel selection, for example might lead 
to faster or more effective "payoffs" in improvements in per- 
sonnel performance. In short, the stimulus to analyze perform- 
ance problems critically in search of specific alternative so- 
lutions might be one of the truly revolutionary effects of pro- 
gramed instruction. 

OVERALL MARKET PROSPECTS 

It is entirely possible that programed instruction will be the 
catalyst for a thorough and dramatic revolution in the mechanics 
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of teaching. The media available for instruction today television, 
tape recordings, films, self -instructional devices are sufficiently 
sophisticated and powerful to be more than ancillary aids to 
teachers. They are capable of performing functions that histori- 
cally have been performed only by teachers. They could, if we 
wished them to, actually replace the teacher in some of his 
traditional functions and, thereby, drastically modify the role of 
the teacher. 

But, if one views programed instruction as simply a new incre- 
ment in a progression of developments in the technology of in- 
struction, then one is less likely to expect dramatic changes and 
the emergence of a vast new market. History seems to be on 
the side of those who expect an emerging market of moderate 
dimensions rather than an explosive growth to large dimensions. 
Still, one can argue that the needs of the time demand the 
" great leap forward" and that the traditional lag in acceptance 
of innovation no longer can be tolerated. If this latter view 
gains enough advocates, the prophecy of a technological revolu- 
tion in educational techniques could be self-fulfilling. 

Tables 5-1-1 through 5-1-4 summarize some of the gross 
characteristics of the markets for instructional materials. 
Table 5-1-1 reports enrollments and total expenditures in the 
regular schools for the school year 1959-60 and projected en- 
rollments and expenditures for 1969-70. 

More relevant to the question of estimating the market for 
programed-instruction materials, however, is Table 5-1-2, 
which presents estimations of expenditures for teaching aids 
and instructional materials. Table 5-1-2 grew from an analysis 
of itemized breakdowns of total expenditures by public schools. 
Public-school systems (K-12), for example, devote about 1.8% 
of all annual expenditures (including capital outlays) to instruc- 
tional supplies, about 0.6% to the purchase of textbooks, about 
0.3 % to school libraries, and about 2.4% to new equipment (e.g., 
laboratory apparatus, school buses). In aggregate, these ex- 
penditures approximate 5 %. Realistically, only a fraction of 
these expenditures can be considered as available for alloca- 
tion to programed-instruction materials. 

It is important to note that the estimated expenditures shown 
in Table 5-1-2 represent expenditures by schools. Inadequately 
accounted for in the table are expenditures by individual per- 
sons for textbooks. Textbooks frequently but far from 
universally are provided free to elementary and secondary- 
school students. At the college level, students most commonly 
provide their own textbooks. Note also that, in 1957, publishers 
grossed about $260 million from the sale of 158 million text- 
books. If schools devote approximately 0.6% of annual expendi- 
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Table 5-1-1. Enrollments and Expenditures of Regular School 

Programs, 1959-60 and 1969-70 (Estimated) 

Enrollment 1 Total expenditures 1 

(millions) (billions $) 

Program 1959-60 1969-70 1959-60 1969-70 

Elementary (K-8) 
(public and private 
combined) 32.9 39.4 11.9 15.7 

Secondary (9-12) 
(public and private 
combined) 9.1 12.8 5.4 8.5 

Higher education 
(public and private 
combined) 3.5 6.1 4.8 8.2 

Vocational 4,0 5.0 0.2 2 0.3 2 

Other post-secondary 
(private, commer- 
cial, nursing, etc.) 0.6 0.6 0.8 0.8 

Totals 50.1 63.9 $23.1 $33,5 

founded to nearest tenth. Sources given in bibliography. 

2 fticludes expenditures for administration; supervision, teacher, and 
counselor training; and instruction and counseling from local, state, and 
federal funds. Estimates of capital outlay unavailable. 

tures to the purchase of textbooks as previously noted, this means 
that only about one-half of total textbook sales are made directly 
to schools. Thus, the total expenditure for instructional materi- 
als used in the regular schools was more nearly $400 million in 
1959-60, and could be as high as $590 million by 1969-70. 

It also should be emphasized that the estimates in Table 5-1-2 
do not presume a decrease in the fraction of the total budget al- 
located to instructional staff. More than half of total current 
educational expenditures are personnel costs. If programed- 
instruction procedures were to replace teachers in any signifi- 
cant numbers, it is apparent that funds available for programed- 
instruction materials could increase radically. We doubt that this 
is likely to happen, at least within the next 5-10 yr. But were it 
to occur, it would reverse a long-term trend in pupil-to-teacher 
ratios there were 37 public-school pupils per teacher in 1900, 
30.4 pupils per teacher in 1930, 27.6 pupils per teacher in 1950, 
and 27 pupils per teacher in 1960. 

Although the estimates presented in Table 5-1-2 are admittedly 
approximate, they almost certainly are more reliable than any we 
have been able to develop with reasonable effort for expenditures 
for instructional materials in military training. Education and 
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Table 5-1-2. Estimated Expenditures for Teaching Aids and 
Instructional Materials by Regular School Programs , 
1959-60 and 1969-70 

Estimated annual 

expenditures 
(millions $) 



1959-60 1969-70 



Elementary (K-8) 

(public and private combined) 1 120 160 

Secondary (9-12) 

(public and private combined) 2 80 130 

Higher education 

(public and private combined) 3 35 61 

Vocational 4 24 30 

Totals 259 381 

Assumes 1% of total annual expenditures for teaching aids and ma- 
terials. 

2 Assumes 1.5% of total annual expenditures for teaching aids and ma- 
terials. 

3 Assumes $10 per pupil per year expended for teaching aids and ma- 
terials. 

4 Estimated from costs of illustrative materials listed in Misc. Pub. 
No. 3243, List of Instructional Materials for the Supplementary Train- 
ing of Apprentices and Other "On-the-Job" Trainees, U.S. Dept. of 
Health, Education, and Welfare, Office of Education, 1958. 

training expenditures by the armed forces are a matter of public 
record, but they typically are reported as aggregate totals rather 
than as detailed breakdowns. In fiscal year 1959, the armed for- 
ces budgeted a total of $1.165 billion for all specifically designated 
education as shown in Table 5-1-3. 

The numbers shown in Table 5-1-3 include such activities as 
the service academies, ROTC programs, armed-forces colleges, 
and institutes, service schools and extension-course programs, 
and various other programs of general, specialized, or technical 
nature for armed-forces civilian and military personnel. Break- 
downs of total budgeted expenditures by item probably could be 
developed from subsequent records of performance. In the source 
from which these data were drawn, however, subtotals for each 
activity are qualified by notes that state, in effect, that a break- 
down of the obligations for the educational and training programs 
into operating and administrative costs is "not feasible." 

It seems doubtful that the percentage of these budgeted totals 
allocated to instructional materials is greatly different from the 
practice of the civilian schools. If the allocation is as high as 
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Table 5-1-3. Education Programs of the Department of Defense 

Education and training budget* 
Agency (millions $) 



Dept. of Defense and 
Joint Chiefs of Staff 


3.2 


Dept. of the Army 


309.3 


Dept. of the Navy 


482.2 


Dept. of the Air Force 
Total 


370.2 


1,164.9 



*Rounded to nearest tenth. Source: Federal Educational Policies, 
Programs and Proposals, Part II: Survey of Federal Educational Acti- 
vities, Prepared for the House Committee on Education and Labor, 86th 
Congress, 2nd Session, June, 1960. 

5%, the armed forces might have expended $58 million for in- 
structional materials in 1959-60. We have assumed 3% to arrive 
at our estimate of expenditures for instructional materials of $35 
million. 

Expenditures for education and training in business and indus- 
try arealmost literally anyone's guess. It seems safe to say 
that no one actually knows how much is spent annually for train- 
ing in industry. (We will be able to report a more complete esti- 
mate of this by next year if current plans materialize for a sur- 
vey of training in industry to be conducted by Stanford Research 
Institute.) Probably the key problem in arriving at a reliable 
estimate (over and above obtaining the information from a depend- 
able industry sample) is in defining accurately and unambiguously 
what is to be considered as "training." 

Clark's estimate of $10 billion* works out to an average annual 
expenditure of about $135 for each person in the total labor force 
or about $170 for each person in manufacturing and service indus- 
tries. Other estimates are more restrained for example, $4 bil- 
lion per year but the fact remains that no estimate can be either 
verified or refuted with any data we have been able to find. How 
much does industry expend for instructional materials and teach- 
ing aids? We have estimated an average of $3 per worker in man- 
ufacturing service industries or a total of about $180 million. 

The precise magnitude of the home-study market for instruc- 
tional materials is unknown. A reasonable but undocumented 
guess is $60 million per year, based on the assumption that 



*Clark, Harold F. and Ruth E. Sobkov (1959) How much can the people 
of the United States afford to spend on education?, Unpublished report. 
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60 million of us spend an average of $1 each year for materials 
to help us in systematic home study. 

Our aggregate guess of annual expenditures for instructional 
materials and teaching aids in major market sectors is summar- 
ized in Table 5-1-4. As our discussion has emphasized, many 
of these numbers can not be documented and all are estimates. 

Table 5-1-4. Summary of Estimated Expenditures for Instructional 
Materials and Teaching Aids by Major Users, 1959-60 and 1969-70 

Estimated expenditures* 
(millions $) 



Major users 1959-60 1969-70 

Regular schools 

(elementary, secondary, 
higher, vocational; pub- 
lic and private combined) 260 380 

Business and industry 180 240 

Armed Forces 35 50 

Home study 60 70 

Totals 535 740 

^Estimates rounded to the nearest $ 5 million. 

What share of this overall estimated total annual expenditure 
for instructional materials might be devoted to programed- 
instruction materials that are carefully developed to be respon- 
sive to the needs and wants of various potential consumers? 
Boehm* has estimated that as much as $100 million will be 
spent by 1970. Although this number seems a reasonable one 
in light of the total expenditures for instructional materials 
estimated in Table 5-1-4, 1970 might be too early a date for it 
to be achieved. 

The following arguments persuade us that the growth of the 
annual dollar market for programed- instruction materials will 
be less rapid than the $100 million estimate would require: 

a. The most dependable market sector is the civilian schools 
and, within this category, the elementary and secondary public 
schools. Diversity in the school curriculum is many times 
greater than the variety represented by existing pretested pro- 
grams. The developmental lead-time for a programed- instruc- 
tion course adequate to the requirements of a school year of 
teaching can scarcely be less that 18 months and easily can 



*Boehm, George A. W. (1960) Can people be taught like pigeons?, 
Fortune, October 1960, p. 177 
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be twice that long. It is likely to be at least 1965, therefore, be- 
fore much real choice is possible among programs with compar- 
able coverage, objectives, and reliability. 

b. Many practical issues are unresolved. The present diver- 
sity in types and styles of devices and programs permits a 
potential user to pick and choose, but it also prevents him from 
interchanging a program that is available for one style of device 
for another program designed for another kind of device. (This 
particular difficulty could be alleviated by a strong trend toward 
relatively cheap-say $5 to $15-devices or programs that re- 
quire no devices such as programed textbooks.) Further, a 
prudent buyer of programed-instruction materials hardly seems 
likely to invest in devices before he can have some assurance 
that additional programs will be forthcoming. A "programing" 
industry, similar to the textbook or test-publishing industry, 
might emerge, and there are signs that the publishing industry 
itself will attempt to meet the challenge. In addition, an in- 
creasing number of teachers at all levels might learn to pro- 
gram. But neither of these developments seems likely to reach 
full flower in much less than a decade. 

c. Effective resistance to the automation of instruction is a 
critical imponderable. Causes of resistance to the adoption of 
an innovation such as programed-instruction are likely to vary, 
but all of them will serve to delay wide -scale adoption. Some 
resistance will come from those opposed to change in the exist- 
ing order. Empirical evidence is not likely to exert much in- 
fluence on people of this sort. Others will resist adoption, on 
other than an experimental basis, until more of the facts are 

in and reliable estimates of cost and "payoff" can be developed. 
For these people, resistance could change to enthusiasm if the 
facts were persuasive. Historically, innovation has come 
slowly in education, be it in the regular schools, industry and 
business, or the military. For programed-instruction to her- 
ald a major revolution in the technology of instruction that 
results in wide-spread adoption rather than scattered islands 
of innovation, the traditional lag must be reduced sharply. Ten 
years seems like a short time for this. 

PROGRAMED INSTRUCTION IN SCHOOLS 

Elementary Schools 

Elementary schools have yet to feel the impact from applica- 
tions of programed-instruction procedures. It is estimated that 
less than 20% of the presently available programs for programed- 
instruction devices apply to elementary schools. Over the short 
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run, one can expect that an increasing number of programs will 
be developed for elementary instruction in basic skills such as 
reading, composition, mathematics, and foreign languages. Over 
the longer run, experimentation likely will occur in subjects 
more difficult to program such as social studies, literature, art, 
and music. 

The first programs to appear generally at the elementary- 
school level probably will be geared to the characteristics of 
the typical, literate student in the middle and upper elementary 
grades. But increasing attention almost certainly will be given 
to programing material for learners with special character- 
istics preliterate children in the primary grades, children 
who are particularly gifted and for whom acceleration or en- 
richment is difficult under the guidance of a single teacher, and 
remedial programs for students who need to catch up or in- 
crease their ability in basic skills. Evidence appearing from 
research conducted at UCLA under Keislar, for example, sug- 
gests that effective programed-instruction courses can be 
developed for instructing preliterate children in complicated and 
complex subjects. Keislar has enjoyed some success in using 
pictures and nonverbal symbols to teach natural-science con- 
cepts to primary-level children not yet able to read. 

At present, the use of programed-instruction materials at 
elementary levels is essentially experimental. It seems wholly 
likely, however, that, within the next 5 yr, programed-instruction 
procedures will be routine in many but far from all elementary 
school systems. 

One important contribution that programed-instruction pro- 
cedures seem uniquely capable of making at the elementary level 
is that of supplementing the background knowledge of elementary- 
school teachers. For example, one trend in curriculum planning 
at the elementary level is toward an increased emphasis on sci- 
ence and foreign language and earlier introduction of fairly com- 
plex mathematical concepts. Elementary-schoolteachers often 
are lacking in preparation in these areas. It seems reasonable, 
therefore, that well conceived pretested programs designed to 
supplement the teacher's resources would be welcomed in 
elementary schools that are willing to innovate and experiment. 

A critical and unanswered question with regard to programed- 
instruction for elementary- school subjects is that of the avail- 
ability of programs. As already noted, very few programs are 
now available. The future progress of programed instruction 
at the elementary-school level will be limited by the rate at 
which elementary- school teachers and curriculum specialists 
familiarize themselves with and become enthusiastic about 
the possibilities of programed instruction. 
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It seems likely that simple mechanical devices will be used at 
the elementary- school level. The programed textbook format, 
although suitable to middle and upper-level elementary-school 
instruction, are probably less effective than programs presented 
by simple devices. It is conceivable that whole school systems 
eventually might move toward the installation of computer- ' 
controlled systems accommodating multiple student stations. 
This prospect is remote for the next five and probably ten- 
years and appears not to offer a feasible system for any but 
quite large school systems. 

Elementary schools last year spent about $120 million on in- 
structional materials of one kind or another, and one could ex- 
pect that some of this will be allocated to programed materials 
and teaching machines. Moreover, it is clear that the element- 
ary-school level represents an important frontier for the de- 
velopment of programed-instruction materials because so little 
experimental effort has been expended in that direction thus far. 
It is not clear who will take the initiative. One guess is that 
textbook publishers experienced with developing materials for 
elementary schools will recognize the opportunities and begin 
to exploit them. 

Secondary Schools 

Wide-scale adoption of programed-instruction procedures is 
likely to come more quickly at the secondary-school level than 
at the elementary- school level. This forecast is based, pri- 
marily, on the fact that the majority of programs currently 
available have been written for secondary-school students, 
college students, or adults. Additionally, subject fields tend to 
be more compartmentalized and somewhat more amenable to 
programing. In keeping with trends in curriculum revision and 
experimentation, one can expect that the earliest applications 
will be in the sciences, mathematics, and foreign languages. 
As is the case of expected developments at the elementary- 
school level, one can also anticipate an increasing amount of 
experimentation with programing in the social studies, humani- 
ties, art, music, literature, and in vocational skills. 

The first programs to be used widely at the secondary-school 
level probably will be for average-ability students, although 
some experimental efforts will be aimed at students needing 
remedial or accelerated work of varying types. One can expect 
gradual development of special programs for students with 
particular handicaps. 

Over the short run, the use of programed-instruction mater- 
ials in the secondary schools will be mostly experimental. 
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Wide-scale adoption could lead to important shifts in school- 
facilities planning and to rather profound changes in the way in 
which audio-visual departments and school libraries are organ- 
ized. School planning increasingly is taking account of develop- 
ments in new media of instruction, apparently in anticipation of 
rather widespread adoption over the next decade. 

Many of the same factors that restrain rapid adoption of pro- 
gramed instruction at the elementary-schools level also apply 
at the secondary-school level. First and foremost, very few 
programs are available. Among the more ambitious develop- 
ment efforts is that of Encyclopedia Britannica Films, which is 
producing, and will soon offer for sale, complete program 
"packages" in science, mathematics, and foreign languages. 
Harcourt Brace has already produced and marketed a programed 
textbook called "English 2600". Materials from other sources, 
such as the Center for Programed Instruction and Teaching 
Machines Inc., also are appearing on the market and can be 
expected to find increasing use in classroom experimentation. 

Over the long run, unless an increasing number of teachers 
and curriculum specialists learn to prepare and refine programs, 
the possibilities for expansion will be severely limited. An in- 
creasing number of workshops and short-course sessions are 
being offered to teachers and others in the "art" of programing. 
As more teachers develop an ability to program effectively, it 
might be that an organization such as Educational Development 
Corporation, serving as a link between author and publisher, 
will be in a position to accelerate the availability of an in- 
creasing variety of programs for regular school use. 

A number of other factors might inhibit the widespread adop- 
tion of programed-instruction procedures. For example, overt 
resistance by teachers, pupils, administrators, school boards, 
or the community in general can not be assessed accurately in 
advance. It is likely, however, that adoption of self-instructional 
procedures in secondary, as well as elementary, schools will 
be spotty and will be concentrated in those districts and areas 
that have a reputation for innovation and experimentation. 

Colleges and Universities 

The outlook for programed instruction at the college level is 
much the same as that at the secondary-school level. One im- 
portant difference is apparent, however. Although colleges and 
universities are interested in applying programed-instruction 
procedures to classroom use, they are perhaps even more con- 
cerned with the research possibilities of teaching machines and 
programed procedures. Consequently, one can expect that most 
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research-minded schools and departments of education and psy- 
chology will, in the very near future, establish laboratories and 
centers for systematic experimentation in programing and the 
learning process. 

Most of the significant research in programed instruction and 
teaching machines is occurring in the colleges and universities. 
Such institutions as Harvard, University of Southern California, 
University of California at Los Angeles, Columbia, Hamilton, 
Earlham, and Oberlin have housed pioneering efforts in program 
development and active experimentation. It is likely that serious 
research with teaching machines is occurring in at least 100 
colleges throughout the country. 

Programs developed in college settings have concentrated, 
initially, on those subject fields that seem most amenable to 
programingmathematics, some engineering courses, certain 
of the sciences, elements of philosophy (particularly logic), and 
language. In estimating the extent to which programed-instruc- 
tion procedures will be adopted on a regular basis in colleges 
and universities, an important unanswered question concerns 
the degree of active resistance that will be forthcoming. There 
are indications, for example, that all faculty members do not 
perceive programed instruction as a blessing. The same can 
be said for student reactions. 

From a marketing viewpoint, it is important to note that most 
of the programs and devices thus far employed in colleges and 
universities have been produced locally by the teachers and 
researchers who are themselves directly interested in learning 
more about the capabilities and limits of programed instruction. 
There has been some sharing of programs among faculty at 
different institutions and certain programs have found their way 
to commercial production and distribution. In spite of the ex- 
ceptions, the rule thus far is that interested faculty members 
have produced and experimented with their own programs. From 
the standpoint of learning more about the capabilities of pro- 
gramed learning, this trend, of course, is all to the good. 

Graduate Schools 

Little has yet been done or if done, has not yet been adequately 
reported concerning the application of programed- instruction 
procedures to graduate professional training. Many aspects of 
instruction in medical schools, dental schools, schools of 
nursing, schools of law, or graduate schools of business lend 
themselves to wide-scale experimentation with programed- 
instruction procedures. The differences between instruction at 
the graduate level in fields such as these and the problems of 
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continuous learning faced by professionals after leaving the 
university setting are so slight that one could also predict the 
developments of professional-level programs for postgraduate 
"refresher" instruction. 

Vocational Schools 

The utility of programed-instruction procedures in the teach- 
ing of vocational and technical skills is almost self-evident. The 
field provides a useful transition from applications in school 
programs to developments in business, industry, and the mili- 
tary. Some of the earliest programs developed covered such 
topics as computer arithmetic and techniques of computer pro- 
graming. Other programs, developed primarily under military 
auspices, covered electronics maintenance and trouble-shooting 
procedures. These, as well as a host of other topics, are ap- 
propriate to much of the instruction that occurs in vocational 
and technical institutions, as well as junior colleges, throughout 
the country. 

Procedural fixed-sequence skills, as well as contingent se- 
quences, generally lend themselves to programing. These pro- 
grams might take the form of step-by-step demonstrators, such 
as, for example, one of the Videosonic systems developed by 
Hughes Aircraft, or might utilize film strips or other pictorial 
displays (e.g., television) offering models of sequential steps 
in an assembly process. Such basic vocational and technical 
skills as key-punch operation, typewriting, shorthand, blueprint 
reading, and the like can also be programed. 

Successful experience with programs in vocational and tech- 
nical fields of relatively high occupational demand might lead 
to the penetration of the home-study market. An interesting 
possibility is the development of systematic testing procedures 
required for granting credentials or certificates of completion 
to people who successfully complete home-study programs. 

PROGRAMED INSTRUCTION IN BUSINESS AND INDUSTRY 

Industry is both a potential user of programed-instruction 
procedures and a producer of programs and devices. From a 
use viewpoint, the promise of programed-instruction materials 
offers much to many industries. For example, those industries 
that experience high labor turnover can gain much by substitut- 
ing programed-instruction procedures for more traditional 
methods of training. Many training and orientation activities 
can be programed fairly simply, including routine assembly 
or repair procedures, sales techniques, or familiarization with 
product lines. Examples of basic skills that can be programed 
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for self -instruction in an industrial setting are almost limitless 
and include shop mathematics, use of tools, parts nomenclature, 
safety procedures, preventive maintenance, and sequential and 
contingent assembly procedures. Some companies with regional, 
national, and international activities are reporting success with 
self-instructional programs that reduce the need for periods 
of centralized training with their high per-diem costs. 

Programed-instruction procedures also suggest a number of 
possibilities for making training more efficient and effective 
in industries characterized by heavy capital investment per 
worker. Programed-instruction procedures seem well suited 
to the type of rigorous and continuous training demanded of 
operators of complex equipment with which the costs of errors 
are high and supervision necessarily must be particularly 
diligent. 

Industry is a producer of instructional programs and devices 
as well as an actual and potential user. Most of the earliest 
program activity has centered in industries that have seen both 
the opportunities for use of such materials as well as the longer 
term prospects of their commercial development. Examples 
include Hughes Aircraft, IBM, Graphlex, Recordak, and 
Minneapolis-Honeywell. Instrument manufacturers also can be 
expected to develop and produce programed-instruction pro- 
cedures and devices. 

Another industry that is becoming increasingly active in the 
development of programs is the publishing industry. Wiley, 
McGraw-Hill, Harcourt Brace, Doubleday, Prentice-Hall, and 
Grolier are examples of publishing firms that have actively 
entered, or have announced intentions of entering, some as- 
pect of the programed-instruction field. One could also expect 
that firms specializing in the development and marketing of 
psychological tests, audio-visual materials, and instructional 
materials generally would make some efforts to enter the 
programed-instruction materials market. 

In considering the future of the industrial development of pro- 
gramed-instruction materials programs and devices it is 
important to recall that the device is secondary and subsidiary 
to the program, A teaching machine does not actually teach. 
It is, rather, a means for presenting the program and accept- 
ing responses from the learner. An early infatuation with the 
prospects of a 'device market" could lead to a misinterpreta- 
tion of the needs in the field. Metal fabrication, plastics, or 
electronics firms that anticipate entering the teaching-machine 
field through the production of devices will soon discover that 
they need to develop, in addition to their fabrication skill and 
mass-production ability, a capability for programing. Alterna- 
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lively, they might enter into a complementary relationship with 
some other firm or agency to provide programs for their de- 
vices. A specific illustration of such a relationship is offered 
by the recently announced agreement between Litton Industries 
and Prentice-Hall. In this instance, Litton will be producing 
devices and Prentice-Hall will serve as a general source for 
programs. 

PROGRAMED INSTRUCTION FOR HOME STUDY 

An additional major market area of unknown future potential 
can be grossly identified as the home- study market. Here one 
finds perhaps as many as 60 million people who are interested 
in self-development and the learning of new skills or the pur- 
suit of particular hobbies. Grolier's marketing of teaching 
machines and programs, through their tie with Teaching Ma- 
chines, Inc., represents probably the first, and certainly the 
most dramatic example to date of an effort to tap this potentially 
large market. Content areas for programing are as broad as 
people's interests in learning or polishing new skills languages, 
hobbies, work supplementary to regular school instruction, 
games and, perhaps, even psychomotor skills. The prospects of 
programing correspondence courses is also a very real 
possibility and can be expected to be undertaken with some vigor. 

If increasing interest in programed-instruction materials in 
the home is forthcoming, it is also possible (and perhaps likely) 
that public libraries will serve as a depository for instructional 
programs and as a checkout point for instructional devices 
that are too costly to be owned by a single family. In this way, 
a family might borrow or rent a device for use and check out 
programs from the library much as they now check out books. 
It is also possible, of course, that a rental and checkout pro- 
cedure could be commercialized, particularly where device 
costs are prohibitive. 

A further possibility with implications far beyond the home- 
study field lies in merging the mass audience capability of 
television and the highly individualized feedback characteristic 
of programed instruction. The major technical problem to be 
resolved appears to be that of developing an effective two-way 
feedback channel between "instructor" and "pupil". 

PROGRAMED INSTRUCTION FOR THE 
MILITARY AND GOVERNMENT 

Applications of programed-instruction procedures in the 
military and government are not dissimilar from the applica- 
tions possible both in civilian schools and business and industry. 
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It seems likely that uses in the military will take advantage of 
more complex instrumentation than is likely to be found in the 
schools, at least in the near future. The military already has 
considerable experience in the use of quite complex and sophis- 
ticated programed-instruction materials and has pioneered in 
their development. The Air Force, for example, has been con- 
ducting for some time an experimental program in the instruc- 
tion of electronics maintenance personnel through programed- 
instruction procedures. The Navy also has been active, and 
the Naval Training Devices Center has been a focal point for 
some of the most interesting and important developmental 
studies that have thus far been reported. 
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Present and Emerging Needs for Job 
Training 

A STAFF MEMORANDUM 
Basic Systems, Incorporated 



This research memorandum examines the present and emerg- 
ing needs of American industry for job training at wage -roll, 
technical, and management levels. A statistical profile of the 
growing demand for specialized and skilled personnel is developed. 
A possible systems facility, incorporating diverse training capa- 
bilities, is outlined, with reference to its potential relevance for 
the training of industrial employees. Although defense industries 
are included in this survey, the internal training needs of the 
military establishment, while great, are not studied. It is con- 
sidered by the authors that the same training-systems facility 
could operate with efficiency in the military area. Although anno- 
tations have not been made in this edition, the documentation has 
been developed from the materials indicated in the attached biblio- 
graphy. 



PART I: Survey of Industrial Training 



STATISTICAL OVERVIEW 

The several billions of dollars now being spent annually by 
business and industry on job training will undergo a substantial 
increase in the near future. In the next decade and a half, the 
labor force will increase faster than the population: from 73.6 
million to 95 million an increase of 34%. This is a greater in- 
crease than in any other 15 -year period in U.S. history. It means 
that, by 1975, one out of every two workers will have entered the 
labor force during the previous 15 years. 

At the same time, the proportion of skilled jobs is rising even 
more rapidly. Within the total labor-force increase of 34%, the 
number of white collar workers will increase by 57% and the num- 
ber of blue collar workers by 25%. In this latter group, the lar- 
gest relative growth will occur in the skilled worker category: 
from 311.5 million. 

Among the professional and technical groups, 5.25 million ad- 
ditional persons will be needed because of national growth, and 

161 



162 MARKET ANALYSIS AND ECONOMIC CONSIDERATIONS 

five million more as replacements. Professional and technical 
personnel represent the fastest growing group in the postwar 
period During the next 15 years it will increase 2 1/2 times as 
fast as the labor force. For example, 650,000 scientists (or 
double the present number) will be needed in 1975, as will two 
million engineers (as opposed to the present number of 850,000). 
This, of course, will condition a very rapid rise in supporting 
technicians, such as engineering aides and electronic specialists. 

In the craftsman group, 2.5 million additional individuals will 
be needed because of total national growth, and 5.5 million for 
replacement. (Example: electricians need to increase by 190,000 
from growth, by 200,000 as replacements.) It is very doubtful 
that present training activities can meet this need, e.g., present 
apprenticeship programs would train only 10% of the eight million 
total required. 

In the sales-worker group, a two million increase over the 
four million individuals employed in 1957 will take place. These 
individuals will need instruction in sales techniques, and the in- 
creased complexity of equipment and introduction of new ma- 
terials will demand even more technical training. The number 
of clerical personnel will increase from 7.2 million to 13.7 mil- 
lion despite advances in automation. 

In terms of age groups, however, there will be a decrease in 
the number of male workers between 25 and 44 years old. This 
intensifies the total training problem since this age bracket us- 
ually supplies a large proportion of foreman, highly skilled 
workers, middle management personnel, and technicians. This 
means, of course, that workers outside of this bracket will have 
to be trained or retrained for a greater proportion of these im- 
portant jobs. 

The U.S. Department of Labor estimates that by 1965 we are 
going to need 137 professional and technical men, 122 managers 
and officials, 127 clerical and sales people, and 122 skilled 
craftsmen for every 100 we have today (see Table 5-2-1). Al- 
though our population will increase by 18 million people in 1965, 
the greatest proportion of this increase will come in the age 
groups that are above 44 and under 25 years of age. Accordingly, 
there will be no increase in the number of men who are in what 
the Department of Labor calls the "prime" working ages between 
25 and 44. 

There is at present no national survey of training establish- 
ments in the United States, although the Bureau of Labor Statis- 
tics has begun work on one. There is, however, a survey of in- 
dustry in New Jersey, undertaken as a pilot project of the national 
survey. It should be possible to extrapolate a good deal from 
these figures, given in Table 5-2-2. 
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Table 5-2-2. Survey of Training in the Industry of New Jersey 

Size of company 
(number of 
employees) 4-19 20-99 100-499 >500 

Number of companies 
surveyed 26,886 8,484 1,857 319 

Percent of companies 
of this size 70 20 5 < 1 

Number of companies 
with training 2,972 1,947 920 241 

Percent of companies 
with training 11.1 22.9 49.5 81.5 

Percent of companies 
with training depart- 
ments 4.2 17.4 17.7 41.2 

Percent of companies 
with training by 
foreman-supervisors 42.2 13.5 31.5 38.1 

Percent of companies 
with training by per- 
sonnel manager or 
staff 52.9 55.0 46.1 19.5 

Out of a total New Jersey employment of 728,531 by companies 
sponsoring training for production, about 70%, or 514,316, were 
employed in manufacturing establishments. These establishments 
represented approximately 24% (9,419 out of 37,416) of all com- 
panies. 

Within the manufacturing group, 241 out of 9,419 companies- 
or about 2% employed 500 or more individuals. This 2% em- 
ployed a total of 315,408 individuals, or 60% of the total of 514,316 
employed in the manufacturing group studies. Thus, while repre- 
senting less than 6/10 of l%of the total, these 241 manufacturers 
employed about 42% of all workers. Of this group 82.6% sponsor 
some form of employee training. 

At the time the survey was made, 21,764 employees were en- 
rolled in training programs of varying duration in the group of 
manufacturers employing 500 or more people. This was about 
50% of all industrial trainees in the state. 1,207 training opera- 
tions were used by this group, an average of about 18 employees 
per operation. 

Out of a total of 728,531 employees (11.3%) employed by 6,099 
establishments sponsoring training (16.2% of all industrial estab- 
lishments) 82,740 employees were currently participating in train- 
ing programs. These establishments sponsored 15,873 training 
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programs. With the average number of trainees per program at 
five, present methods appear uneconomicalparticularly from a 
systems standpoint. 

More than half of all establishments supplemented internal 
training with that offered by some outside facility. 41.5% of all 
firms used no outside facilities, and 58.6% of the manufacturing 
group used no outside facilities. 68,740 employees were currently 
enrolled in 11,813 in-company programs, an average of about 6% 
per program. 

In short, a survey of this typical manufacturing State reveals 
that job training is today almost nonexistent among smaller com- 
panies, though their need for it, as can be shown, is great; that 
training procedures, in those companies where they do exist, are 
poorly rationalized, usually involving minuscule student popula- 
tions at any one time; that companies already committed to train-- 
ing are the large ones which, as other analyses show, usually 
have sufficient funds to support more efficient methods as they 
become available. In terms of input-output analysis particularly 
in view of the total amount of training actually required the pre- 
sent approach appears uneconomical and ill designed. Whereas 
human teachers and conventional techniques are excessively 
costly for small and scattered student groups, programed in- 
struction in a behaviorally designed learning context promises 
far more efficient and economical employee training. 

REPRESENTATIVE TRAINING PROBLEMS 
High -Turnover Jobs 

A number of important job categories requiring mastery of 
specialized subject matter are characterized by very high turn- 
over. For example, there are more than 196,000 life-insurance 
agents in the United States who are employed full time. In addi- 
tion, there are 100,000 property and casualty insurance brokers. 
Turnover is estimated to be as high as 70% annually. At the same 
time, most companies maintain elaborate, lengthy training pro- 
grams for these men and there is expected to be still greater 
emphasis on in-service training of agents. This pattern extends 
to other job categories such as registered securities representa- 
tives, mutual fund salesmen, airline clerks, door-to-door sales- 
men, bank service personnel, lower management, etc. The sub- 
ject matter taught in each case is relatively static and involves 
working with large student population. 

Retraining Following Automation 

Retraining will probably continue to be increasingly significant 
in the coming 15-year period and often involves skilled and semi- 
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skilled workers displaced by automation in continuous process 
production. Attempts to retrain these workers are being carried 
on by industry and by government. 

An example of industry effort is the Armour & Co. program 
worked out with the Amalgamated Meat Cutters and Butchers, 
and the United Packinghouse Workers. Armour allocated $500,000 
for a program to train "qualified employees in the knowledge and 
skill required to perform new and changed jobs so that the pres- 
ent employees may be utilized for this purpose to the greatest 
extent possible/' Symptomatic of the inadequacy of the total pro- 
ject, however, was the fact that 65% of the displaced workers ap- 
plying for the program were eliminated by screening tests, as 
not possessing adequate prerequisites for the training involved. 

Under a section of the recently passed Federal depressed areas 
bill, money is provided for the organization of retraining projects 
on a community level. Projects are under way in Hazleton, Pa., 
and Lowell, Mass. Much more comprehensive provisions for re- 
training are contained in the Clark-Holland bill. Under its terms, 
the Federal Government will pay up to 100 per cent of the cost 
to the States. Under terms of agreements worked out between 
Federal and State Governments, State agencies may use public 
education facilities or, if such facilities are not adequate they 
may arrange for training with private institutions. 

Commenting on the bill in a statement to the House Committee 
on Labor and Education, HEW Secretary Ribicoff said: 

"...technological change, innovation, automation and indus- 
trial mobility displace many people who want to work and 
who are physically and mentally able to work but who can- 
not work because they lack the knowledge and skills which 
they need. Moreover, it appears that these changes ... will 
increase at a much more rapid rate than they have in the 
past ... 

"We must shift toward more training for technicians, distri- 
bution, sales, personal and community services and other 
areas required by our national manpower needs ... 
"Of the five million unemployed, about one -third are under 
age 22 and half of these haven't held a job since leaving 
school. Most are unskilled. 

"The older worker constitutes a special need for training 
and retraining. As of April, 1961, the Department of Labor 
reported that 31% of all unemployed were over age 45, and 
35 of those unemployed for 15 or more weeks were over age 
45." 
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Apprenticeship Programs 

Apprenticeship programs exist today mostly in older crafts 
and industries. But even among such groups as plumbers, print- 
ers, machinists, etc., this system will provide only about 10% of 
the craftsmen needed by industry in the next 15 years. 

Often, apprenticeship systems are financed by an employer or 
industry associations, in collaboration with trade unions. Wide- 
spread adoption of apprenticeship programs has been inhibited 
by their cost to management despite the fact that they provide an 
assured supply of skilled craftsmen for the company sponsor. 
Apprentices must be paid at a very high rate to maintain their 
motivation to finish programs which in most cases run from 4-8 
years. Because of its length, apprenticeship strengthens worker 
ties with the company and cuts down on turnover. Nonetheless, 
learning time seems excessive by even the most inefficient in- 
structional standards. 

This relatively small aspect of industrial training accounted in 
1958 for only 178,000 young people in programs meeting the 
standards of the Federal Bureau of Apprenticeship. In contrast, 
the Soviets claim to have trained 7.5 million young workers as 
skilled hands in 500 narrowly specialized trades during the last 
15 years; in 1955 alone, they turned out 92,000 engineering tech- 
nicianswhich is seven times as many as we trained in the same 
year. Clearly, job training even for older skills must be over- 
hauled, shortened, and made more effective. 

Continued Training for Professional and Management Groups 

According to the U.S. Office of Education, Bachelor's degree 
graduates may reach a total of 10 million in the 1960-75 period. 
It is estimated that professional and technical degrees will ac- 
count for 60 to 75% of this total. Thus, 3-4 million professional 
and technical jobs will have to be filled by persons without col- 
lege degrees. This clearly implies that existing programs for 
upgrading skills within industry will be greatly enlarged and many 
others initiated. 

Of all professional groups and, in fact, of all occupational 
groups in the labor force engineers will require the most exten- 
sive and continuous training. In 1958 there were approximately 
850,000 engineers in the United States, three-fourths of them em- 
ployed in private industry. (Those employing the largest numbers 
were electrical equipment, aircraft and parts, and machinery in- 
dustries.) It has been, and will continue to be, one of the fastest 
growing occupations. Losses from retirement and death were 
10,000 in 1958 and are expected to gradually rise in the next 15 
years. 35,332 Bachelor's degrees in engineering were granted in 
1958 and this number is expected to double in the 1960's. 
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The need for continual training of this key group of personnel 
has already been recognized by management and teaching authori- 
ties in the field. Dr. Chester J. Kishel, head of the Industrial En- 
gineering Department of Fenn College in Cleveland, noted that 
engineers fresh out of the classroom are commanding high sal- 
aries because those who graduated a decade ago are obsolete, re- 
marking: "For an engineer to keep up to date on developments 
he would have to spend 30% of his time in study." 

Mr. John Phillips, of DuPont's engineering division, offered a 
similar comment. He said effectiveness of engineers now in their 
thirties and forties is severly reduced by lack of knowledge and 
competence in new technological developments. Schools run by 
computer companies, for example, are filled with industrial en- 
gineers catching up with new developments. 

Although it is estimated that the managerial group will show 
the smallest rate of increase in the 1960's, there will be a three 
million increase over the present force of 6.7 million. As in the 
case of other occupations, this group will need to possess far 
more knowledge of a far broader kind about a widening variety 
of processes and situations. Beyond maintaining competence in 
technical areas, executives, of course, need knowledge of develop- 
ments in business organization, economics, labor contracts, and 
still more general disciplines. 

Continuous Training for Other Job Categories 

With the increasing number of new products and processes, 
most workers will continue to be trained throughout their work- 
ing life. The electronics industry is a striking example. There 
were 170,000 electronics technicians employed in the electronics 
industry in 1958, and this number is rapidly expanding. (In addi- 
tion, 250,000 members of the armed forces are trained in elec- 
tronic maintenance.) The need for thorough training extends be- 
yond this skilled group to assemblers who are constantly con- 
fronted with new designs. Short product runs, involving entirely 
new technical problems at frequent intervals for an increasing 
labor force, intensify this problem. While many companies now 
attempt to meet the training need, others, with substantial dollar 
volume but small work forces, would like to do so if costs could 
be rationalized. In the closely related field of computer manu- 
facturing, greater efforts are being made in the face of a similar 
problem. All computer companies have substantial training pro- 
grams; IBM leads with an annual education budget estimated to 
be over 45 million dollars. Similarly, the Bell System and Wes- 
tern Electric complex maintains continuous training for most of 
its 770,000 employees. 

Industrial salesmen or sales engineers sell mainly to other 
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business concerns. These individuals, primarily men, require 
exhaustive knowledge of their own firm's products and almost 
equally comprehensive knowledge of the operations of the client. 
These salesmen may spend weeks or months analyzing client op- 
erations and working with his own company's research depart- 
ment to find client needs which his products can meet. Further- 
more, salesmen are frequently called upon to train client's em- 
ployees in the use of his firm's product after sale is made. Many 
corporations have extensive training programs for such salesmen 
which last two or more years. 

Pharmaceutical companies, for example, have a critical need 
for instruction to enable their detail men to master the complexi- 
ties of new drugs and biochemical preparations. This is doubly 
vital because it has been estimated that 80% of physicians' knowl- 
edge about new drugs comes from the medical detailer. Producers 
of primary metals, machinery manufacturers, plastics suppliers, 
and the industrial chemical industry, all need salesmen who have 
a substantial knowledge of their own product lines and the proc- 
esses of client firms. The salesmen for computer manufacturers 
are a prime example. Other areas of electronics need high qual- 
ity technical selling in a range that comprehends components for 
missiles and jet aircraft to parts for tractors and washing ma- 
chines. 

Maintenance technicians keep complicated equipment represent- 
ing a high capital investment on an operational basis. Accelerat- 
ing technological complexity puts an unprecedented demand upon 
them for both scheduled and unscheduled maintenance. As George 
Baldwin and George P. Schultz of the MIT Industrial Relations 
Department have suggested: 

"...the new technology takes over most readily the mate- 
rials handling and completely routinized machine opera- 
tions and tends to emphasize, as far as the average plant 
workman is concerned, jobs directed at 'keeping the proc- 
ess going because we just can't afford to chase all over the 
factory for a maintenance man when something goes wrong. 
He's got to be right there and he's got to know something 
about electrical and hydraulic problems, not just mechani- 
cal.' So the proportion of maintenance people is likely to 
increase as well as the skill required of them. ..Mechaniza- 
tion may indeed have created many dull and routine jobs; 
automation, however, is not an extension but a reversal of 
this trend: it promises to cut out just that kind of job and 
to create others of higher skill... The training or the edu- 
cational job impliedwill obviously become more difficult 
and more important as the speed of innovation increases... 
For example, when three or four types of grinding opera- 
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tions, each now representing a separate job classification, 
are tied together by automation, one man will be able to 
operate the integrated grinding line. This man must have a 
generalized knowledge of grinding; and his changed, broader 
job classification is likely to carry more pay than any of the 
old grinding occupations... As a standard for continued em- 
ployment, 'ability to learn' would gradually replace 'ability 
to do ? the job." 

The introduction of automated machinery in production, elimi- 
nating human efforts at the base level of operators or assembly 
workers, but increasing demand for increasingly skilled mainte- 
nance technicians, is being felt in food and beverages, paper and 
allied products, tobacco processing, petroleum, chemicals, as 
well as in industries producing the automated equipment itself, 
such as machine tool and electronics. A DuPont executive, con- 
firming the MIT report, remarked that plumbers, electricians 
and mechanics have been replaced in chemical production by 
"process mechanics" who can function across established trade 
lines. 

The efficiency of present industrial maintenance is difficult to 
estimate, for few figures are available. However, military ex- 
perience with similarly complex equipment may be relevant; Hall 
has shown that only 33% of Navy electronic equipment was operat- 
ing satisfactorily in 1950, and that 9% of electronic equipment in 
Fleet Units was inoperative and 65% operating poorly or impro- 
perly in 1955. The problem of millions of failures was serious 
not only because of the cost in billions of dollars, said Hall, but 
also because of the decrease in operational effectiveness. Among 
examples he cited were certain types of complex airborne elec- 
tronic equipment requiring over three hours of maintenance for 
each hour of flight; that one -third of Air Force operating cost is 
for maintenance and one -third of Air Force personnel are engaged 
in this activity, although over half of Air Force maintenance is 
done by contractors. Additionally, it was pointed out that the cost 
of maintaining electronic equipment in the Air Force was about 
twice as much per year as the initial cost, that maintenance dur- 
ing the life of Navy equipment is ten times initial cost. While 
maintainability can, in some instances, be dealt with at the design 
stage, "unscheduled" problems are not alleviated by such techni- 
ques as self-test diagnostic -routine circuits and check-out fault- 
isolation programs. 

The makers of automated equipment found that maintainability 
of electronic components, systems and subsystems involves many 
factors, among the more significant being systematic job analysis, 
equipment design for maintainability, realistic technical training, 
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practical job instruction devices and high-speed field modifica- 
tion information communications. At the 1956 Air Force Fire 
Control Systems Training Conference, it was demonstrated that, 
while the maintenance of complex electronic systems was not be- 
yond the learning ability of most trainees, imparting these skills 
to the learner was fast-exceeding the teaching abilities of the in- 
structors. 

Preparation of Manuals 

About 2.8 billion dollars are spent each year for the prepara- 
tion of manuals. The "Bell System Procedures" are an example 
of a comprehensive set of materials covering, in minute detail, 
specifications for every component, device and task found in the 
total technology of the telephonic communication industry. In de- 
sign and use, however, present industrial manuals appear to 
compromise unhappily between training needs and the require- 
ments of reference works. Much like textbooks in academic train- 
ing, industrial manuals are ambiguous in function. They combine 
elements of a "rule" book, of a teaching implement, and of an 
encyclopaedic compendium of facts and figures. Much of the 
money now budgeted for manuals is likely to be allocated to di- 
rect training as the need for efficiency in skill dissemination is 
more widely felt. Job manuals are now invariably written by en- 
gineers and scientists who are the creators of the equipment de- 
scribed, or by professional manual writers who are, by education, 
lower -level technicians; the manuals are therefore usually poorly 
organized from the standpoint of learning psychology, and fail to 
recognize the special needs of operating and maintenance person- 
nel. These materials could be vastly improved, and given greater 
performance for each dollar spent on their preparation, if learn- 
ing psychology were systematically applied. 

A CASE STUDY: TRAINING PROGRAMS IN A LARGE 
ELECTRONICS CORPORATION 

The representative training problems just described and others 
are best seen in the specific model of a working industrial organi- 
zation. We here describe the training efforts involved in the sale 
of equipment of one out of four divisions in one group of the cor- 
poration's domestic operating divisions. The division employs 
about 5,000 persons, 3,000 in the laboratory, 2,000 in the factory. 

On-Line or In-House Training 

This training effort is headed by six administrators who super- 
vise the efforts of 40 instructors, who have no other function in- 
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side the company. Most of the instructors are part time; all of 
them are from outside the company. 

Training Categories 

The following are the two major job-category breakdowns: 

a. Administrative. Executives, managerial and supervisory 
personnel. The lowest category here, that of Supervisor, is sal- 
aried at about $18,000. 

b. Technical Engineering Personnel, Technicians. This group 
is composed of technicians without a degree who need to have 
thorough and advanced engineering training despite their job 
classification as wireman, trouble shooter, etc. Included above 
are draftsmen who prepare, develop and modify drawings in the 
laboratory, and shop personnel in various craft categories. 

It is estimated that about 80% of personnel in training are in 
group b and about 20% in group a. All students are paid full rates 
for time in training, It is estimated that the average salary in 
group a is three times that in group b. 

Company Courses 

About 150 courses-all formal classroom training are given 
to both groups. The administrative group takes courses entitled 
management training, executive development and supervisory 
training which consists of such subject matters as management- 
labor interaction, preparation of company forms, report writing, 
administrative and company standards, etc. These courses are 
mandatory. The technical group studies advanced technical sub- 
jects, mostly at an engineering curriculum level. A number of 
courses are given all employees when they first enter the com- 
pany. These are mandatory. The rest are voluntary. 

Courses in Universities and Technical Institutes 

Any employee of the corporation can take any course or degree 
for which he is qualified and which the company approves as rele- 
vant. The company allows the student 3-4 hours a week with pay 
for this study. It also reimburses him for all tuition, 50% paid 
when the study is begun and 50% when he receives the degree or 
completes a given curriculum. 

Direct Training 

This training is conducted by the division as part of the con- 
tract with a client, which requires that the company be respon- 
sible for adequately trained personnel to operate the product. 
Products of this division are various kinds of electronic systems. 
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The division meets this responsibility by furnishing personnel 
from its own staff, by hiring outside personnel or by training per- 
sonnel from the client company or Armed Forces unit. All the 
personnel from whatever source have to go through a training 
system designed to provide the human capabilities to combine 
with the electronic system belonging to the group. In performing 
this job, the division combines with another division of the cor- 
poration whose exclusive function is the preparation of training 
systems, including preparation of manuals, etc. 

Training of Instructors 

In direct training efforts the first task is the training of instruc- 
tors who will in turn train the personnel who will operate and 
maintain the electronic system. This group is formed from the 
technical staff of the division (designers, engineers, etc.), or is 
hired from outside the corporation. 

School Facilities 

School facilities are set up for each training effort, sometimes 
using client's facilities, sometimes those of the division. 

Scope and Number of Such Training Efforts 

The training division of the defense group is at present handling 
training for four military contracts. These all involve systems 
for the Air Force devoted to early warning, integrated communi- 
cation, control systems, etc. One of these training efforts is bud- 
geted at $6,000,000. These programs are devoted solely to teach- 
ing technical personnel how to maintain and operate various parts 
of the major and minor subsystems which make up a system. 

Training Project 

Following is the example of the organization of a training proj- 
ect for a major subsystem which is one of three major subsys- 
tems and two minor subsystems which make up one system of the 
four presently being handled by the training division. Total cost 
for this single subsystem is budgeted at $250 to $400 million. 

a. Training of instructors. Eighteen instructors. Training 
time: 10 months. Subject matter: Operation and maintenance of 
subsystem. This involves highly advanced categories as of elec- 
tronic knowledge at engineering level. 

b. These instructors then train 320 persons in this case all 
military personnel. 

c. This training involves some simulation but is primarily 
classroom training, academic in nature. Objective of the training 
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is to create experts with very broad backgrounds in electronics 
and also specific knowledge of the subsystem structure who can 
perform the trouble -shooting function which is highly creative, 
conceptual and not a rote task. Also, a good deal of computer 
programing is taught. 

d. In this case, simulation involves the use of a giant computer 
group and accessories, creation of which is itself a major job. 
This simulation of the actual system is then used both to train 
personnel and to locate areas where electronics must be adapted 
to human factors in the subsystem. Subsequently, adjustments 
are made in the actual subsystem. 

e. Manuals are written, of course, prior to the above training, 
but they, too, are sometimes changed in response to experience 
with them in the training process. 

. A large number of courses are given which take from six 
weeks to six months. 

A Sample Student and What He Must Know 

The student's job will be transmission equipment maintenance 
supervisor. The Air Force Specialty Code requires that he have 
15 years training and experience in transmission equipment, its 
installation, maintenance and operation. He must understand 
electronics, binary mathematics, computer programing, pulse 
techniques and logical and digital circuitry. He must be familiar 
with transmission equipment design and modification. He must 
be thoroughly experienced in installation, operation and mainte- 
nance of transmission equipment such as Modems, Cryptos, auto- 
matic exchange equipment, leased line switching equipment, fault 
and facilities control equipment, line unit equipment, clock sig- 
nal generating equipment, etc. He must be able to operate all the 
kinds of test equipment that are used with the above. 

It is extremely unlikely that this type of trained person is found 
already prepared. He must be given training covering the above 
which includes: Intermediate electronics, Boolean algebra, digi- 
tal and pulse circuitry and complete orientation in the parts in- 
volved in the subsystem both as to operation and maintenance. 

Critical Nature of Systems: Reliability Factors 

All of these systems are built at least in duplex. If they are 
complicated to operate and maintain, and breakdowns are pro- 
jected, then they are built in triplex or quadruplex and so on de- 
pending on degree of frequency and length of repairs. If training 
can increase human factors efficiency in maintenance, the sys- 
tems duplication can, of course, be reduced. A reduction in one 
system from quadruplex to duplex eliminates $38,000,000 for the 
computers alone. 
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The maximum time allowed for a major subsystem to be not 
available or not on the air in this case is 1/10 of 1% . About 10% 
non -available time is allowed for one in a quadruplex system. 
The mean time allowed for repair of a part in electronic military 
systems of this kind is under one hour. 

Location of Systems and Resulting Salary Scales 

Many of these systems are located in remote regions such as 
the polar region. Civilian personnel who receive $10,000 a year 
in the United States are paid about $27,000 in these areas. Fre- 
quently, it is impossible to provide military personnel with ade- 
quate prerequisite training and civilians must be hired. 

Training of Salesmen 

There are 15 salesmen for the division. They are distributed 
over the country. Their chief function is information gathering. 
This requires them to have a very broad technical background 
and to be able to analyze and report on systems needs in the mili- 
tary and industry and to give specifications which the designers 
and engineers can follow. 

These men need training, first to fill out gaps in their techni- 
cal background and keep them up to date; second, to enable them 
to gather the information adequately, in the proper sequence and 
quickly enough to meet lead times. These salesmen or contract 
men are, of course, essential to the business. They are paid in 
a range of $13,000 to $15,000 a year. 

Language Training for Administrative and Engineering Personnel 

Fluency in various languages would be most helpful and useful 
to this particular corporation as it has many overseas divisions. 
Language training could usefully be given to executives, supervi- 
sors, managers, engineers, etc. who are both American and for- 
eign nationals. 



PART II: Behavioral Learning Design for Industry 



OPTIMAL LEARNING DESIGN 

Programed instruction, as has been repeatedly demonstrated 
and as has been documented in the scientific literature, offers a 
rapid, flexible, economical and efficient solution to the problems 
described above. Tested applications of learning theory delineate 
the following elements of efficient curriculum design: 
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a. Creation of a formal response mode to ensure active learner 
participation. 

b. Presentation of material in discrete units with deliberate 
and small increments forming a cumulative sequence. 

c. Feedback in the form of response evaluation immediately 
following each micro-unit. 

d. The instructional cycle is, except for special cases, self- 
pacing. 

The usefulness of constructing such sequences on an empirical 
basis was first fully stated by B. F. Skinner in 1954. Many of 
Skinner's conclusions were prefigured in the experimental founda- 
tion layed by researchers such as Pressey (1926) and the still 
earlier work of experimental behaviorists such as Thorndike. 
Intensive research along the specific lines suggested by Skinner 
took on an institutional pattern in the twelve years following 
Skinner's first paper. Research into programed instruction has 
received support from Air Force and Navy laboratories, the Ford 
and Carnegie Foundations, and the United States Office of Educa- 
tion; experimental projects have been carried out at nearly 200 
schools and universities. The final impetus of popular interest, 
having been inspired by display-device manufacturers with a 
somewhat distorted emphasis on device characteristics rather 
than on curriculum design developed during 1960, although full 
commercial exploitation was still a year or two away. One of the 
first areas to suggest itself for full scale application of pro- 
gramed instruction is business and industrial training where the 
advantages of the new method spell themselves out with particu- 
lar consistency and relevance. In this context, where goals are 
specific and benefits measurable, the feasibility of programed 
instruction can easily be demonstrated and its introduction great- 
ly facilitated. 

Universal Advantages of Programed Instruction 

Programed instruction provides a mastery of subject matter 
that defies comparison with the results of any other method of 
instruction and greatly increased speed of learning. Programed 
instruction, though still in an imperfect stage of development, 
reduces learning time from 50 to 33% of that consumed in tradi- 
tional methods of instruction. Rigorous evaluation examinations 
reveal major improvements in skill mastery and comprehension. 

With programed instruction, training costs can be reduced and 
accurately measured. There is presently no objective and practi- 
cal way to measure the efficiency of a particular textbook or in- 
structor. A program can be subjected to rigorous tests on a 
variety of sample populations. Its cost is explicit and can be 
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averaged or amortized like any other cost. It is not hidden in 
overhead or spread out over part-time efforts of supervisors, 
engineers, or complicated deals with outside universities, voca- 
tional schools and technical institutes. Radical cost reductions 
can be achieved. With present training techniques, a very limited 
number of employees can be trained at any given time. This 
means that the operating costs and overhead of instruction is 
very high per worker especially if there are many different spe- 
cialists. Obviously, programed instruction could cut these costs 
and isolate them. 

Programed instruction overcomes geographical separation of 
trainees. Trainees who are spread out geographically either in 
separate divisions of a corporation of in sales territories do not 
have to be transported and maintained at company expense in 
some central training center. Programs can be mailed to them 
and tests administered by division managers. 

Programed instruction reduces extensive screening operations. 
With programed instruction, the extensive screening operation 
now undertaken by personnel departments of many corporations 
can be reduced. Easier and more efficient training can make up 
for deficiencies in technical knowledge. This also allows execu- 
tives responsible for hiring to employ applicants they feel are 
strong in other areas such as personality and attitude. 

Programed instruction eases the shortage of good instructors 
and reduces instructional cost. In industry as in the schools, there 
is a shortage of teachers and an extreme shortage of good ones. 
In his description of the organization of the Lockheed training 
program, Karl R. Kunze, the company training director said: 
"Lockheed found itself in a position comparable to neighboring 
school systems it needed 'education' and lacked educators. The 
skilled man who could instruct was needed in two places at once 
his own work station and the training class." Such problems can 
now be overcome through specially constructed programs. It 
would also seem that far more thorough and extensive training 
could be given, and the supervisory load eased, with programed 
instruction. Companies where no training department now exists 
would especially benefit. Training departments themselves could 
be reorganized, their cost reduced and their results precisely 
measured with programed instruction. 

Programed instruction enlarges the content of training pro- 
grams. As a corollary, companies would have the option of greatly 
increasing the content of programs and depth of subject matter 
taught. Standard programs could be devised which would be eco- 
nomical for smaller companies employing less than 99 employees 
and their training programs thus greatly extended and strength- 
ened. 
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Programed instruction adjusts to individual differences in 
ability formal education and job categories. Differences in de- 
gree of ability, formal education and job category divide indus- 
trial employees into a wide variety of student groups-far more 
than exist in school populations. In addition to these differences, 
seniority gives promotional priority in many cases to older 
workers who tend to have a lower level of formal education. In 
some cases, union contracts freeze men into jobs whose descrip- 
tions have been radically upgraded by technological change, (i.e. 
electronic assemblers). Many companies follow and others would 
prefer to follow a policy of promotion from within to maintain 
high morale. They are frequently prevented from doing so because 
employees lack knowledge and skills. Many industrial employees 
have been educated or have educated themselves informally in a 
process of work experience, correspondence school courses, in- 
struction from supervisors, moving from job to job, etc. Even 
those possessing a roughly identical background, vary widely in 
their knowledge of and mastery over a given subject matter. 
Again Mr, Kunze: " Another problem had to do with the wide 
range of individual differences in ability to learn among those 
who entered the early training programs. These differences were 
so great that the brighter and more capable were unmotivated 
and others much less gifted were confused." 

Industry Awareness of Programed Instruction 

Public awareness of the field of programed instruction was 
stimulated by a number of popular presentations appearing in 
magazines such as Time, Saturday Evening Post, Newsweek, 
Pageant, the New York Times Magazine, and others. At the same 
time, of course, specialized pieces and monographs continued to 
appear in the field; emphasis on applications to business and in- 
dustrial training grew with the appearance of articles in such 
periodicals as Industrial Relations News, Fortune, Journal of the 
American Society of Training Directors, Fire Engineering, Indus- 
trial Research, Factory and Personnel Journal, This generated 
increasing interest in the practical use of programed instruction 
so that workers in the field received frequent and numerous in- 
quiries. Seminars and speakers throughout industry and business 
have begun to address attention to the new technology. Institu- 
tional bodies such as the American Management Association or 
the Electronic Industries Association have been giving extensive 
consideration, through conferences and special committees, to 
many of the topics detailed in this memorandum. Several of the 
largest industrial, among them DuPont, IBM, Eastman Kodak, 
Hughes Aircraft, Polaroid, and General Dynamics have initiated 
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studies and experimental tests which have given them a fairly 
sophisticated grasp of, and an affirmative attitude toward, the 
new instructional system. Companies in close touch with the 
military are naturally conscious of an emerging technology which 
has been heavily funded and closely examined by the defense es- 
tablishment. 

Company funds and company training programs will not be 
committed to programs, however, on any full scale before pros- 
pective users have received practical demonstrations of the 
method's merit. Probably the most convincing approach is to 
organize a small-scale pilot project within the given company. 
Promotional materials, citations of success elsewhere, scientific 
documentation, are all much less persuasive. 

Even initial resistance to automated training need not be signif- 
icant if the introduction and demonstration are made in a series 
of prudent, controlled steps. Such an experimental study might 
be approached in the following: 

1. Undertake an analysis of the efficiency and expense of cur- 
rent training procedures in order to compare them with pro- 
gramed job training. 

2. Choose a finite area of job training providing a satisfactory 
number of experimental trials. 

3. Divide the trainees into control and experimental groups 
(and perhaps, in addition, a "blind" group which is not told of its 
place in the experimental context) of similar composition. As- 
sessment of trainees should be maximally detailed for purposes 
of accuracy in evaluation. Predictor or control variables should 
be isolated and measured prior to the training schedules for use 
in covariance adjustments. (Suggested data-gathering tools might 
include: IQ estimates based on the Otis Employment Test espe- 
cially Test 2, form B or the Wechsler Adult Intelligence Scale; 
scores on the Ohio State University Reading Comprehension Ex- 
aminations; Bennett's Test of Mechanical Comprehension; Pace's 
High School Characteristic Index; Stern's Activities Index; subject 
matter aptitude indices such as the Air Force Electronics Apti- 
tude Index; and specific examination results on pre -knowledge of 
the course material. For this latter, one might use selected items 
from course tests. With this battery it should be possible to test 
for and weigh such factors as latent learning and sensory precon- 
ditioning.) Naturally, no single experiment need employ more 
than a few such instruments; the listing merely indicates the 
range of the armamentarium. 

4. At first, experimentation should be limited to the simplest 
forms of display device or programed text. 

5. Criteria variables, to be studied at completion by all groups 
of the programed material, would be: time (number of hours of 
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work on program or in conventional study to completion); test of 
factual knowledge (items based on objectives of course); test of 
conceptual facility (use of course concepts in varying problem - 
contexts); costs (of program preparation and devices, amortized, 
as against teacher time and student time, monitoring personnel, 
classroom space, etc.); student reaction (positive or negative at- 
titude expression toward programed teaching as opposed to con- 
ventional or previously experienced training, willingness to con- 
tinue with program as measured at various intervals before com- 
pletion, interest in further such study as assessed after comple- 
tion, to be measured through interviews or questionnaire -polling.) 

In summary, if programed training is introduced flexibly, in 
measured steps, with convincing demonstration of its merits at 
each stage, the new technique can gain authority and acceptance. 
Large sums of money are involved: the client company and the 
programing groups as well can only proceed with certainty if the 
proper base is built for the innovation. A possible cause of delay 
in use of programing, and the factor that can contaminate its repu- 
tation, is the situation described by Levine and Silvern in a re- 
cent paper: "...fantastic claims, marketeers, a sudden outbreak 
of 'consultants 7 , gimmicks and gadgetry without experimental 
evidence in support of claims, and finally, a rash of untried cur- 
ricula/ 7 

If, on the other hand, cautious pilot projects multiply, industry, 
which is already eager to learn about the new teaching technology, 
will develop both sophistication and enthusiasm. Sales are pre- 
dictable thereafter. 



A NEW INDUSTRY RISES 

The complex training needs of industry, scarcely less than the 
educational requirements of the school system, offer vast fields 
for the effective development and introduction of programed 
learning. However, it is likely that the behavioral expertise nec- 
essary for creating good instructional programs and optimal 
learning design is not yet fully recognized in either industrial or 
academic areas. Some industrial companies and public school 
systems have begun highly preliminary experiments in develop- 
ment of their own programed courses; but it soon becomes clear 
that effective programed learning requires a greater concentra- 
tion and integration of specialized scientific ability than achool 
systems can afford or industries can normally find economical. 

In view of these facts, new companies specializing in the de- 
velopment of programed learning have been increasingly emer- 
gent within the past year. Moreover, some of these companies 
also are beginning to serve as academic and industrial consult- 
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ants in overall learning design. Thus, programs in a growing 
range and variety of academic fields are now in publication or 
preparation from several sources and, expectably enough, at 
several levels of size, quality, pedagogical depth, and subject- 
matter coverage. Industrial programs supplied by professional 
companies can be expected to keep pace in development with aca- 
demic ones. 

To be truly effective, the programing company requires large 
capitalization; a staff of experts not only in behavioral psychology 
but also in as wide a range and variety of subject matters as can 
be afforded; expert analysts of industrial training situations 
authorities capable of detailed job and task analysis, examination 
of the total industrial organization, and redesign of the learning- 
work situation in whole or in part. Such a company must also 
have access to academic consultants at all relevant levels and the 
highest attainable degrees of quality. 

The evident complexity, care, and expertise required for reli- 
able high-quality programing and learning design will, in all 
probability, gradually discourage most industries and schools 
from attempting to appropriate the preparation of programed in- 
struction to themselves. Thus companies specializing in pro- 
gramed training and behavioral learning research and technique 
not only may be expected to play an important role in the very 
near future, but also may well rise to a dominant position as con- 
sultants and creators of educational materials generally. 

Systems of educational design are most efficiently constructed 
on an empirical basis, employing tested principles of behavior 
control, rather than "cookbook" procedures. A number of models 
for alternative systems exist, from programed manuals and sim- 
pler man-machine symbioses to multidimensional looped struc- 
tures involving computer mediation. 

An approach flexible enough to achieve effectiveness under va- 
rying conditions demands a constellation of systems factors func- 
tioning as an integrated facility. Given a sufficiently broad spec- 
trum of means, a major transformation of present training and 
education, yielding unprecedented advantages, becomes feasible. 

The following are the four indispensable and distinct elements 
in such a system: 

a. Programing and behavioral technology. Here, the central 
requirement is for scientific expertise, both methodological and 
substantive, together with access to major research centers. 

b. Publication and dissemination. Here, major needs are for 
size, in the sense of a broad marketing range; soundly structured 
relationships with schools and local governments; fluency and ex- 
perience in consumer merchandising; technical efficiency in the 
manufacture of learning materials. 
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c. Instrumentation and hardware. In this area, the needs in- 
clude production and design capacity for a large family of devices 
ranging from display mechanisms and sound equipment to simu- 
lators, closed-loop psychomotor units, and still more specialized 
instrumentation. Complete familiarity with electronics systems 
and computer potentials is necessary. 

d. Special situation sales and marketing. The required char- 
acteristics in this case are a high level of competence in inter- 
action with business, industry, the military, and nonmilitary gov- 
ernmental agencies; access to decision-making levels of manage- 
ment and command; a fluent and articulate sales approach fully 
utilizing relevant technical information; and the capacity to anti- 
cipate emerging markets and potential users. 

The normal activity of each member component can predictably 
derive significant advantage from an interlocking structure of 
this kind. Additionally, new tasks and new opportunities can be 
generated through a systems approach to the total educational 
and training market, 

STAGES IN A SYSTEMS APPROACH TO A 
LEARNING ENVIRONMENT 

An optimal approach to creating the learning environment in- 
volves first the calibration of goal gradients. In other words, es- 
tablishing job performance criteria. These performance stand- 
ards dictate the design of the training process and are estab- 
lished by analysis. Determination of goals and subgoals can us- 
ually be achieved on an empirical basis, through available work 
information; the job or task analysis which grows out of such per- 
formance criteria may also modify them. For example, this may 
mean that in some cases, a set of clearly written job instructions 
can reduce the place of training. The performance standards are 
subsequently deepened into a full task analysis by the joint efforts 
of technical experts and behavioral psychologists. From this task 
analysis, curriculum specialists and learning methodologists are 
able to construct the course outline, or syllabus. After identifi- 
cation of student characteristics, such as aptitude level, and de- 
termination of the degree of generalization required, flow charts, 
including lesson-plans, can be developed on a realistic basis. 
This last step is necessary so that a mosaic of functions can be 
rationally arranged for maximum response differentiation, thus 
avoiding intervening variables. This is clearly seen in the case 
where successive approximations leading to the desired terminal 
behavior are considered as simple analogues, and the optimal 
teaching strategy differs from the anticipated course outline. On 
this basis, a program is constructed, reviewed, tested, rewritten, 
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tested, revised, repeatedly until all validating criteria, in turn 
are derived from the initial job performance standards, are 
satisfied. 

Presentation characteristics of programs may vary consider- 
ably. The simplest case might involve the use of programed texts 
only, or certain general -purpose display devices handling stand- 
ard-form materials. At this level, instrumentation is purely 
auxiliary but may be desirable for reasons of economy, employee 
morale, or through a need for performance records. More highly 
specialized display devices, auto-paced, with audio and/or visual 
components, may be necessary where these differentiations are 
integral to the training objective. One example involves the train- 
ing of sonar operators to discriminate between signals and ran- 
dom noise and between different types of signal, achieving a more 
sensitive interpretation than is possible through reading the visual 
stimuli, on a screen, into which the initially received sounds have 
been translated. For this purpose, sound-discrimination training 
might employ not only aural stimuli but also their visible forms, 
as on an oscilloscope. Still more complex teaching machines will 
be necessary in the case of optimal sequence training, where 
manipulative operations such as maintenance skills require in- 
struction in calibration, inspection, adjustment, repair and re- 
placement as part of a spectrum of trouble -shooting strategies. 
The learning environment in such cases must simulate in detail 
the eventual operational environment, reducing behavioral oscil- 
lation to a minimum, since "downtime" with major capital equip- 
ment and continuous processes is enormously expensive. In still 
another range of cases, machines might supervise as well as 
teach, where errors are costly and the cost of quality-control 
supervision is high. Still other tasks will call for closed-loop 
psychomotor devices which might, for example, pace the student 
by adjusting speed to test the maximum capability of the learner. 
Any one or more of the above devices may be computer mediated, 
where factors such as multiple -input or high-speed evaluation 
are desired. As current limits in machine versatility are ex- 
ceeded, further generations of devices will become possible. It 
should be noted that this entire technology is learner-centered, 
in contrast to instructional methods which, lacking active student 
participation, are organized for maximum teacher convenience. 

Variant Learning Environments 

Research and practical experience will suggest still wider ap- 
plications for learning technology. Viewing sales situations as 
teaching transactions, for example, has suggested the use of psy- 
chological knowledge about verbal behavior in sales training. 
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Sales personnel trained in the control of an interview through its 
communication variables will be able to maximize the advantages 
available in these brief interactions. Knowledge gained in these 
areas, and in the related field of simulation, may evolve an ulte- 
rior usefulness so that even such general topics as marketing be- 
come susceptible to psychological analysis, permitting the con- 
sumer to be treated as a student whose behavior can be predict- 
ably shaped, 

The Range of Services 

The existence of such a complex, and its use of a systems ap- 
proach, permits the solution of training problems of any magni- 
tude. Along the spectrum, these might vary from such tasks as 
discriminating the threads of a screw, or color coding transistors, 
to the informed survey of factors affecting management decisions; 
from teaching the ability to read technical language or engineer- 
ing drawings and translating them into job action by means of a 
programed manual to designing a total training system for a given 
industry. A total training program must be structured in modular 
units covering every aspect of training. For example, an indus- 
trial chemical company might want all its employees to know 
more about the basic science underlying the industry or some 
special chemical field such as polyester resins. Using express 
stops, branching and segmentation, the same program could be 
used at every level of the job hierarchy and could be combined 
in whole or in part with courses in other subjects to fit individual 
needs. The company would be able to control precisely the de- 
gree of training given each individual. It could limit the training 
or remove all limitations to progress to higher levels. A total 
system of this kind, utilizing the fullest resources of the service 
complex, would involve continuous supply to the client of all 
needed training, at levels of employment from wage-roll workers 
to upper executives. Here, the systems approach means that job 
training is planned contemporaneously with changes in production 
technology. 
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The Industrial Market for Programed 
Instruction 



By WILBUR L. ROSS, JR. 
U.S. Army Personnel Center 



This year, industry will spend roughly $1,000,000,000 to train 
some 800,000 employees. If programed instruction continues to 
achieve the approximately one -third reduction in training time 
that initial installations have indicated, the technique might ac- 
count for a large portion of this vast market for training ma- 
terials. Because the industrial -training market is three times 
as great as the combined school and college textbook market, 
many companies active in programed instruction have announced 
their intention to participate in the industrial market. 

Interest in programed materials is not confined to companies 
selling such materials. Training directors across the country 
have commented favorably on the new medium, and some 20 to 
30 large corporations have already purchased programs and/or 
machines. The American Society of Training Directors (ASTD) 
is actively promoting programed instruction; a year and a half 
ago its monthly journal began a continuing series of articles on 
the subject, and ASTD has devoted many hours of its recent con- 
ventions to discussions of programed instruction. 

Thus, although interest in programed instruction is widespread 
among potential industrial buyers and potential suppliers, there 
is a paucity of published material regarding the nature of the mar- 
ket. This article represents one set of opinions regarding the 
factors that bear on the marketing of programed materials to in- 
dustry. 

FACTORS FAVORING INDUSTRIAL ACCEPTANCE 
OF PROGRAMED INSTRUCTION 

Perhaps the most important single factor favoring rapid indus- 
trial acceptance of programed instruction is cost-plus-fixed-fee 
(CPFF) contracting for government, particularly the military. In 
many of these contracts, speed of production and quality of work- 
manship are rightly more important than precise cost to the gov- 
ernment. By including the cost of programed instruction in their 
bids, manufacturers can experiment with this technique at no cost 
to themselves and, at the same time, achieve significant savings 
in time and improvements in quality of product. Furthermore, 
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assuming that workers' speed and effectiveness are increased, 
the company, in effect, will have increased the productive capa- 
city of its facilities. This increase in capacity will not affect 
margins on individual CPFF contracts, but it might permit the 
company to accept more contracts than otherwise would be possi- 
ble. If so, total profits and return on investment will increase; 
if not, the company will lose nothing because the government bears 
the cost of the experiment. 

Defense contractors are not the only companies that have good 
reason to use programed instruction. Many companies experience 
a strongly seasonal pattern of sales and employment; retailing 
operations peak in the pre-Christmas season, and other companies 
have definite peaks and dips at various times of the year. Because 
of these variations such companies repeatedly must train new 
personnel to perform specific skilled tasks for relatively short 
periods of time. By the time the new employees become skilled, 
the seasonal peak ends. It is impossible to estimate the total dol- 
lar cost of training seasonal employees, but some companies re- 
port that the expense is as much as one -third or even one -half 
the total compensation paid the temporary employee during the 
seasonal peak. Such companies will be favorably disposed toward 
any training technique that promises to reduce training time. 

Programed instruction also will be useful to companies whose 
business does not fluctuate markedly from season to season but 
that typically experience a high rate of product or personnel turn- 
over. High turnover is characteristic of airline stewardesses, 
insurance and mutual fund salesmen, and some key employees in 
other service industries. Although employee turnover in many 
space age industries is low, their product lines are so highly 
technical and subject to such rapid change that production workers 
and salesmen must be retrained repeatedly. The pharmaceutical, 
semiconductor, and office -machine fields are three prime ex- 
amples of industries that -require frequent retraining of person- 
nel because products change so often. 

Customer training is related to the training of company person- 
nel to handle rapidly changing product lines. As American indus- 
try produces increasingly sophisticated equipment, more and 
more emphasis is being placed on training users to gain maximum 
benefit from the devices. In electronic data processing, for ex- 
ample, IBM and other companies have established training cen- 
ters for customer personnel. A similar situation is reportedly 
developing in electronic and ultrasonic test equipment and certain 
types of machine tools. As data processing, information retrieval, 
tape -controlled production equipment, and other young science - 
based industries undergo further development, customer training 
should become an even more important sales tool than it is now. 
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Many students of the business -machine industry already insist 
that IBM's extraordinary growth rate is due, not to the superi- 
ority of its equipment, but rather to the company's ability to help 
customers make the most effective use of data-processing sys- 
tems. 

Programed instruction could be used to train customers, not 
only to use equipment, but also to maintain it. As the tools of in- 
dustry become more advanced in design, they require increas- 
ingly sophisticated and specialized maintenance personnel. Even 
in those cases where the selling or leasing company enters into 
a maintenance contract with the customer, someone must be 
trained to do the work. The importance of training in trouble 
shooting is not confined to complex industrial equipment. Tele- 
vision-receiver, washing-machine, and refrigerator repairmen 
already undergo highly technical training, and, if color or pay 
television and ultrasonic -cooking ranges become commonplace, 
a whole new complex of service problems will materialize. Pro- 
gramed instruction is one of the major possibilities for coping 
with these heightened demands on the skills of repair technicians. 

In addition to job-related training programs, many of our more 
progressive companies have begun to offer to their employees 
courses of more general value. For example, Polaroid employees 
can study basic courses in Spelling and English Grammar. In the 
near future, Polaroid management hopes to offer to its employees 
classes in various foreign languages. Other firms have similar 
plans to help workers expand their educational backgrounds. On 
a slightly more commercial level, most major firms offer man- 
agement-development programs featuring such courses as indus- 
trial relations, economics, marketing, and corporate finance. If 
programed instruction can provide an efficient means of present- 
ing either general or management-development subjects, it will 
undoubtedly be well received by industry. 

Cost-plus-fixed-fee contracts, seasonal business fluctuations, 
high rates of product and personnel turnover, customer training 
in the operation and maintenance of equipment, and employee - 
education programs are major factors that will convince industry 
to adopt programed instruction. But, there are two other situa- 
tions that also might help industrial-programing organizations to 
sell their products. The efforts of the National Association for 
the Advancement of Colored People (NAACP) to secure positions 
for Negro workers is one situation, and the activities of various 
area-development councils is the other. The NAACP has effec- 
tively employed economic boycotts to convince various industrial 
organizations to hire Negros, but has found it difficult to obtain 
high paying positions for these people. One reason for this diffi- 
culty is the Negro's characteristic lack of specialized training in 



192 MARKET ANALYSIS AND ECONOMIC CONSIDERATIONS 

skilled functions. Programed instruction might be a relatively 
inexpensive and rapid solution to the problem. Similarly, many 
states and counties, particularly in the South, have convinced 
businesses to relocate to their areas by offering generous plant 
financing and tax shields. In some cases, however, these area- 
development councils have failed to attract industry because their 
regions lack personnel trained in the advanced skills required by 
atomic -age companies. Again, programed instruction might be 
the answer. 

PROBLEMS IN SECURING INDUSTRIAL ACCEPTANCE 
OF PROGRAMED INSTRUCTION 

Industry clearly needs to improve its training procedures, but 
it probably will not accept programed instruction on a large scale 
until several problems have been solved. Perhaps the most criti- 
cal of these problems is the disparity of backgrounds among indi- 
viduals to be trained in a given subject. Unlike schools, where 
one fifth grader has much the same background for each course 
as does every other fifth grader, there is usually no common 
background among the employees a company assigns to a specific 
task. For example, the roster of a drug company's salesmen 
might include some men who have attended medical school, others 
with no graduate training but an undergraduate background in sci- 
ence, and several with only liberal arts training. Yet, if pro- 
gramed instruction is to be effective, it must be suited to the pre- 
vious experience of the trainee. 

If too many frames of the program are explanations of techni- 
cal terms that many students already understand, students will 
find the program boring as well as too time consuming. But, if 
the program does not explain these terms, it might frustrate those 
trainees who are relatively unfamiliar with the material. The ex- 
tent to which employee backgrounds differ will vary from com- 
pany to company and from subject to subject, but it does seem 
clear that background disparity among trainees will be a signifi- 
cant obstacle to industrial programers. The solution of the diffi- 
culty will probably hinge on the flexibility with which the material 
is presented. Branching (use of remedial frames), skipping (let- 
ting experienced trainees omit parts of the material), tracking 
(preparation of separate programs for trainees with different 
backgrounds), or some combination of these three might be nec- 
essary. Such refinements will increase the time and the expense 
involved in program development. 

Programing is a lengthy and expensive process even without 
such refinements as branching, skipping, and tracking. An indus- 
trial programer probably will not be able to prepare more than 
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three or four frames per hour. If so, and if aprogramer's total 
compensation, including fringe benefits, averages $3.50 per hour, 
the direct cost of industrial programing will be about 90 cents to 
$1.20 per frame. 

Because a typical industrial program might run to 2,500-5,000 
frames (the equivalent of a 50-100 page, conventionally written 
manual), the direct cost of writing the program would range from 
$2,500 to $3,000 for the shorter programs and from $4,500 to 
$6,000 for the longer programs. Allowing for 75% overhead, de- 
velopment costs would total $3,950 to $5,250 for 2,500 frames 
and $7,900 to $10,500 for 5,000 frames. Testing and rewriting 
could increase these values by 25-50%, and if an outside contrac- 
tor performs the work, further allowance must be made for that 
company's profit margin. Finally, reproduction costs must be 
considered, although these will vary markedly with the presenta- 
tion format and the number of copies required. 

On balance, programs produced by a company for its own use 
might cost $5.00 to $7.50 per frame, whereas those prepared by 
an outside company might run to $10.00 or more per frame. If 
the programs are used in conjunction with teaching machines, the 
initial cost might be significantly higher. 

Thus, the cost of even a 2, 500 -frame program will be in the 
tens of thousands of dollars. As a result, high-level-management 
approval probably will be necessary for purchases of industrial 
programs prepared on a contract basis, and decisions, in most 
cases, will be based on strictly economic considerations.* Only 
where programed instruction offers sufficiently tangible savings 
to justify the investment will they be accepted. 

Unfortunately, it is difficult to isolate costs and savings in the 
industrial -training situation. The author has read industrial pro- 
posals that devote several pages to the development of "dollar 
value of savings to be achieved via programed instruction" or 
some similarly labeled sum of staggering proportions. But analy- 
sis of the projected savings usually reveals certain invalid or at 
least questionable assumptions. 

The most common assumption is that programed instruction 
somehow will reduce overhead expenses. If the company nor- 
mally spends for overhead a sum equal to direct-labor cost, the 
writers of industrial proposals often assume that a reduction in 
direct-labor hours will result in a proportionate saving in over- 
head. In actual practice, this usually will not be the case. Most 
components of overhead will remain relatively constant regard- 



*Customer training and CPFF contracts are the major exceptions to 
this statement. 
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less of any reductions in training time achieved by programed 
instruction. 

Similar to the overhead "trap", but more subtle, is the diffi- 
culty of computing the value of the man-hours saved through pro- 
gramed instruction. It is relatively simple to compute the num- 
ber of man-hours saved, but in many cases nothing is gained by 
saving the time. For example, if the foreman now spends several 
hours a day training apprentices to operate a certain type of ma- 
chinery, eliminating this function from his duties will be valuable 
only if the time he saves can be put to productive use. In many 
situations, the foreman would be idle for part of the day if he 
were not training new workers. Management, therefore, should 
carefully determine whether or not the time saved by programed 
instruction will actually improve the annual statement. 

Once management is convinced that worthwhile Savings might 
be achieved with programed instruction, it should next decide 
whether or not the anticipated savings justify the purchase. Man- 
agement should decide upon an acceptable relationship between 
investment and savings. Considering the difficulty of achieving 
the time saving and measuring its value to the company, manage- 
ment probably will demand high rates of return on investments 
in programed instruction. For this reason the author doubts that 
industry will accept programing proposals that anticipate more 
than a one- or two-year payback. 

Just as management probably will prove reluctant to accept 
bids promising long paybacks, it will, in many cases, turn down 
programing proposals because of the long lead time involved. To 
use the same numbers as in the section on the cost of the tech- 
nique, let us assume that aprogramer can create three or four 
frames per hour and that an industrial program ranges between 
2,500 and 5,000 frames. With these assumptions, we find that 
600-800 man-hours are required for the shorter program and 
1,250-1,700 for the longer one. If a single programer were to 
work on a 2, 500 -frame program, it would take him roughly fifteen 
40 -hr weeks. Writing a 5, 000 -frame program might take over six 
man-months. These are long time periods, and in many cases, 
management will be unable to wait while the program is under- 
going development. 

COMPETITION FROM OTHER MEDIA 

A general difficulty that will be encountered by companies at- 
tempting to sell programed instruction to industry is competition 
from other recently developed training techniques.* For example, 

*Any of these materials could be used in conjunction with programed 
instruction, but budgetary considerations make it unlikely that this will 
be the case. 
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Technical Animations, Inc. of Port Washington, New York has de- 
veloped an ingenious method of demonstrating various processes. 
The method known as "Technamation", involves plastic trans- 
parencies that are animated either mechanically or by use of ro- 
tary polarization. A number of leading manufacturing organiza- 
tions, including Brunswick Balke, General Electric, American 
Machine and Foundry, and Bulova Watch have reportedly used 
Technamation successfully in a variety of applications. 

Other sources of competition will be various concerns engaged 
in the production of life-sized simulators. Burtek and Link Avia- 
tion are but two of the leading companies active in this field. All 
of their training aids compete in one manner or another with pro- 
gramed instruction. 

The final and most direct source of competition will be conven- 
tional technical writing companies. The present dollar volume 
for technical manuals is several hundred million dollars, but 
most of the individual companies involved are rather small in 
size, or are minor divisions of large companies. Miles Samuel- 
son, Lytle, Genge, Howard W. Sams, and the Technical-Writing- 
Service division of McGraw-Hill are all active. But, no one firm 
sells as much as $5,000,000 per year of technical manuals. 

HOW WILL THE INDUSTRIAL MARKET DEVELOP? 

As noted above, most of the technical writing companies are 
rather small in size. Technical Animations, Burtek, Link, and 
other firms offering training hardware are also modest-sized 
operations. The author believes that the same conditions that 
have led to the present profusion of small industrial training firms 
will tend to fragment the industrial market for programed instruc- 
tion. 

Several factors are responsible for the present situation. First 
is the tendency of big companies to develop their own training de- 
partments. The existence of large staffs of qualified personnel 
means that most major companies are capable of producing their 
own training materials and devices internally. Industrial firms 
in this position tend to subcontract only those materials that are 
needed on such a large scale or that require such specialized 
creative or production facilities that it would be uneconomical 
for the using company to produce them itself. In other words, 
there is a built-in limitation on the dollar volume that producers 
of training aids can sell to a given company. Once this limit, 
which varies from company to company, has been reached, the 
user probably will perform the work and no longer contract with 
another company for the work. 

There already are indications that many companies with large 
annual training costs are developing their own internal program- 
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ing capabilities. One informed source reports that there are no 
fewer than 27 independent programing efforts underway at IBM. 
Judging by the volume of inquiries received by the Center for 
Programed Instruction and other nonprofit groups, many training 
directors are anxious to develop programing staffs. As knowl- 
edge of programing techniques becomes more widely dissemi- 
nated, it seems likely that the trend toward internal program de- 
velopment will continue. 

Even if the major potential customers of industrial program- 
ing organizations do not develop their own capability in this area, 
the growth of such programing companies is likely to be severely 
limitedby geographical considerations. Most industrialprograms 
will require close coordination between the programer and the 
industrial organization because the material is, in many cases, 
highly technical, and the programer will rarely be a subject- 
matter expert. He must work closely with the using concern, 
therefore, to ensure that he is actually presenting the desired 
material. To have frequent discussions with clients, the program- 
ing organization must either maintain an office in each geographic 
locale in which it hopes to secure contracts or allow in its con- 
tract price for the travel expenses of the men in the field. 

In the opinion of the writer, there is only one way that an indus- 
trial programing company can achieve significant nation-wide 
volume and a proprietary position. This is by preparing programs 
covering material that must be learned by employees of many 
companies, but in which the volume sold to each company would 
not justify that firm's establishing an "in-house" programing 
capability. Programs of this sort would include those dealing 
with the use of various measuring devices, such as micrometers 
and calculating machines, and basic industrial subjects, such as 
industrial relations, economics, basic electronics, mathematics, 
antitrust regulations, and a variety of other topics that recur 
throughout industry. 

There are other, more specialized, types of material that have 
widespread applicability. These programs mainly cover certain 
clerical activities common to many companies within a given in- 
dustry. For example, Learn-A-Max Corp. of Indianapolis has se- 
cured one contract to prepare a program for training bank clerks 
and another for retail-sales clerks. Learn-A-Max was able to 
offer an economical bid price by retaining rights to sell the pro- 
grams to other banks and retailing organizations that, because of 
their geographic location, do not compete with the companies that 
let the initial contracts. Assuming that the programs prove effec- 
tive, Learn-A-Max might well achieve a six- or seven-figure 
sales volume for each program. Furthermore, the nature of the 
subjects probably will not change markedly over the years, and 
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Learn-A-Max might have developed the nucleus of a repeat busi- 
ness similar to that of book publishers with lists of back titles. 

Other programing companies should find it possible to use the 
Learn-A-Max strategy to advantage in many areas. U. S. Indus- 
tries has secured a contract to develop a program for training 
salesmen at Prudential Life Insurance Company, and Hughes 
Aircraft has one for training firemen. Similar subjects that come 
readily to mind include programs for training mutual-fund sales- 
men and jet airline pilots to pass qualifying examinations. 

Programs that have proved successful in presenting widely 
needed subjects should be a continuing source of income to the 
programed-instruction company. Once a program has produced 
satisfactory results for several different companies, it will be 
extremely difficult for a new, competing program to gain a signif- 
icant share of the market because no potential buyer is likely to 
purchase a new and untested program when a demonstrably suc- 
cessful one is available. This built-in market resistance, the in- 
vestment required to prepare a competing program, and the risk 
that it might prove to be no more efficient, or even less efficient, 
than the one already on the market and hence unsaleable, should 
tend to discourage new firms from competing with established 
programs. 
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The current level of interest in auto -instructional techniques 
emphasizes a growing concern that traditional methods of instruc- 
tion might not be capable of implementing the expanding need for 
more effective education and technical training. This interest has 
led to important strides in the techniques of auto -instruction, its 
breadth of application, and the design of accompanying devices. 
Systematic research already completed has, at least partially, 
verified the potentialities of programed instruction and has lent 
support to plans for the eventual wide -scale adoption of teaching 
machines in the classroom and in vocational training programs. 

It has been only recently, however, that it has been possible to 
consider certain of the problems made apparent through extensive 
field studies employing auto-instructional materials. During the 
spring of 1960, an investigation of the use of auto -instructional 
materials in an education situation was conducted by the authors 
under the sponsorship of the United States Office of Education. 
This was the first field study of auto -instruction carried out on 
a large enough scale to require a detailed consideration of cost 
factors involved in the use of these materials. Up to that time, 
only small amounts of auto -instructional materials were avail- 
able, no published programs had been offered for sale, and only 
prototype models of devices had been produced. 

During the course of this study, some 500 students were fur- 
nished with auto -instructional materials consisting of about 3,000 
frames in high-school physics. These materials represented 
slightly less than 20% of the content of the corresponding physics 
course but, nevertheless, involved logistics of considerable mag- 
nitude. For instance, the materials furnished to these 500 stu- 
dents required over 3 1/2 tons of paper and 120 dozen loose-leaf 
notebooks. Had the entire course been prepared in the programed- 
text format used, it would have been necessary to furnish each 
student with the equivalent of twenty two 300 -page volumes at an 
estimated cost of $66 per student. 

These estimates make it apparent that efforts to implement 
auto -instructional research would likely be stymied until the cost 
of auto -instruction could be reduced to satisfactory levels. As 
will be described shortly, an expenditure of the public schools of 
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more than $5 per student per course per year would be prohibi- 
tive at the present time. Even in industrial training, where the 
allowances made for training costs are considerably higher, 
large expenditures for course materials are certain to be met 
with considerable resistance. Apprehensive that the bulk of pro- 
gramed materials being prepared might fail to meet the needs of 
education and training in spite of their technical advantages, we 
decided that a detailed analysis of costs was needed to clarify 
the problems involved. 

In investigating cost factors in auto -instruction, two principal 
sources of expense need to be considered: first, the cost of writ- 
ing or developing a typical program; and, second, the cost of re- 
producing it for classroom or on -the -job training. Certain appli- 
cations of these materials might necessitate additional expenses. 
These applications include programs involving a high degree of 
flexibility or those involving skill learning that could be less ex- 
pensively achieved through the use of a human tutor. To simplify 
the analysis, however, only those programs that warrant general 
distribution were considered. Furthermore, because of the rather 
extensive differences in technique involved in preparing short- 
frame, constructed-response programs (cf. Skinner) as opposed 
to paragraph-length, multiple -choice "intrinsic" programs (cf . 
Crowder), the costs involved in each of these two approaches are 
apt to be substantially different. In this analysis, only the con- 
structed-response type of program is considered. 

THE COST OF PREPARING THE PROGRAM 

To determine the cost of preparing a typical program, certain 
assumptions had to be made. These assumptions are based on ex- 
perience obtained during the preparation of the physics materials. 
First, it is necessary to assume that the personnel responsible 
for preparing the program already have been fully trained. It was 
discovered during the 1960 study that those reasonably skilled in 
psychology, education, and the subject matter being programed 
often required several months to reach satisfactory levels of pro- 
ficiency in preparing frames. 

Second, it is assumed that the developmental procedure in- 
volved in preparing the program includes all the steps that ap- 
parently contribute to the production of successful auto-instruc- 
tional materials. These steps include a reasonable amount of edi- 
torial review, repeated tryouts with individual subjects, diligent 
efforts to revise the program on the basis of student responses, 
and extensive assistance from a subject-matter expert. 

Third, it is assumed that the program being prepared is not un- 
usually difficult or easy to write. The preparation of materials 
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in foreign-language vocabularies or arithmetic, for example, do 
not involve nearly as much effort per frame as materials that 
instruct students in principles and applications. On the other 
hand, it is not assumed that as much time would be spent per 
frame as would be required to program materials such as might 
be involved in problem solving or creative writing. In these sub- 
jects, the subject matter is such that considerable preliminary 
effort is required to determine exactly what skills are to be 
taught and to sequence the teaching of these skills in an optimum 
fashion. 

Fourth, it is assumed that the program is being prepared on a 
production rather than a research basis. It became apparent dur- 
ing the development of the physics materials that a minimum goal 
for programed materials is that they do produce the desired level 
of achievement with a minimum number of student errors. In 
many cases, research programs have the additional aim of pro- 
ducing achievement as efficiently as possible and as consistent 
with a given theoretical position as possible. These latter pro- 
grams require greater effort to prepare, and, although they gen- 
erally can be expected to contribute to an understanding of pro- 
graming, the additional effort will not generally yield proportion- 
ate increases in learning efficiency. 

Fifth, it also is assumed that the program will be of sufficient 
quality to demonstrate the effectiveness of the auto-instructional 
approach and, thus, find acceptance in public-school education or 
in industrial or technical training. 

As a basis for estimating developmental costs, the most suit- 
able standard seemed to us to be the cost per frame. An accurate 
estimate of the number of frames that might be required to pro- 
gram a typical course is difficult to make, however. Not only 
does the estimate depend heavily on the subject matter and the 
ability level of students or trainees for whom the program is in- 
tended, but it also is influenced by the current repertoire of the 
students, by the degree of retention or proficiency aimed for, 
and by the complementary use of other instructional methods. 
As an example, retention probably is closely related to the num- 
ber of times a given response has been practiced in the program. 
The greater retention desired, the greater the number of frames 
needed in the program. 

On the basis of the physics study, a year -long physics course 
would require approximately 15,000 frames. This estimate of 
roughly 100 frames per instructional hour has been substantiated 
in the preparation of other programs at American Institute for 
Research. Although it might be possible to prepare a program in 
high-school physics that consisted of as few as 7,000 or 8,000 
frames, there probably is little to be gained by skimping on the 
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amount of material covered, the amount of review provided, and 
the number of examples used. The figure of 100 frames per in- 
structional hour has been found to be equally applicable in indus- 
trial and military training. Thus, a technical training course of 
about six weeks in duration might involve about 24,000 frames. 

The amount of time required to write a single frame was esti- 
mated by means of a survey of a number of programer including 
Glaser, Evans. Homme, Holland, and Eigen. From their replies, 
it was possible to estimate a typical production rate of between 
one and two frames per hour. This does not represent the amount 
of time necessary to draft a frame but, rather, the composite 
time needed to draft, edit, tryout, and revise frames so that they 
are ready for field tryout.* On the basis of these figures, the 
preparation of a 15, 000 -frame program would require approxi- 
mately 10,000 man-hours, or five man-years in professional sal- 
aries alone. If reasonable allowances are then made for research- 
ing the subject matter, secretarial and administrative support, 
overhead, paying test subjects, and limited duplication of the pro- 
gram, the overall cost could well amount to $75,000, or $5 per 
frame. 

In practice, we have found that even an estimate of this magni- 
tude might be low, depending on the difficulties encountered. The 
most difficult step in preparing a program, like many other en- 
deavors, is in getting started. Thus, a relatively short program, 
or one that requires the development of frames that cannot be 
modeled after previous work, requires a substantially greater 
amount of time per frame so that the cost of preparing a 400 or 
500 frame program might well be as much as $5,000. It should 
be pointed out, however, that costs of this magnitude cannot be 
considered excessive, especially when amortized over several 
years and many thousands of students. If developmental costs of 
$75,000, for example, are prorated over a conservative estimate 
of 30,000 students per year for five years, the basic cost for the 
entire program is only 50 cents per student. This is a sound 
argument for not skimping on frames or on the amount of tryout 
and revision. 

It should be recognized that auto -instructional programing is 
not likely to remain an exclusive art for very long. Teachers, 
writers, and technical experts will soon be preparing programed 
materials as they prepare textbooks today: on their own time and 
with a minimum of expense. While a trend in this direction is 
economically unavoidable, it should be viewed with caution. The 



*This estimate has since been verified by a more extensive survey 
completed by Joseph W. Rigney and Edward B. Fry of the University of 
Southern California. 
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educator working at home, regardless of his skills and good in- 
tentions, is rarely in a position to provide the thousands of man- 
hours necessary to prepare a good course. Tryout of the program 
alone requires considerable effort. Reviewing even one student's 
responses to a 15,000-frame program is a 100-hour job at the 
rate of 2 1/2 frames per minute. Because of the large number 
of man-hours involved, it is not likely that someone working in 
his spare time will be in a position to persist long enough to pre- 
pare a full-length course properly. Even if one were to prepare 
only 8,000 frames and, by omitting much of the tryout and review, 
average four frames per hour, the complete program would still 
require hisfulltime efforts for an entire year. 

THE COST OF REPRODUCING THE PROGRAM 

The second major problem considered in our analysis was the 
expense involved in reproducing a program for student use. Here 
too, several variables, such as the size of frame, the use of illus- 
trations, and provision for automatic scoring affect an estimate 
of costs. It also is important to consider how the program will 
be used. For example, auto-instructional devices ordinarily can- 
not be taken home from school or work, but materials prepared 
in programed-textbook format can be carried about by the student. 

Perhaps the simplest form of presentation is that using stacks 
of cards, programed textbooks, or other formats that require no 
extrinsic devices. The size of frames reproduced on paper can 
vary considerably. The programed textbooks prepared for the 
physics study contained presentation and confirmation panels that 
were about 2 x 7 in. each or 28 in. 2 for the two portions of a 
frame. If five panels are placed on a page, a book of about 6,000 
pages is required. 

It is possible, of course, to reduce the space devoted to each 
frame, but it seems unlikely that the presentation portion of a 
frame can be reduced to much less than 6 in. 2 Because the con- 
firmation panel rarely needs to be as large, there is a tendency 
to double up on panels by placing the confirmation panel next to 
the following presentation panel. This approach reduces the 
amount of paper required by more than one -third, but it has the 
unfortunate disadvantage of potentially contaminating a student's 
response. Whenever the preceding answer is visible to the stu- 
dent, it cannot meaningfully serve as the response to the frame 
being attempted. Thus, it again is questionable whether sacrific- 
ing some of the effectiveness of the program so as to reduce the 
cost of production is a real saving. 

Returning to the physics study, the materials supplied to the 
students in the 25 classes involved in the experiment did not re- 
quire printing on a scale comparable to what would be expected 
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were the program to be distributed for general use. Thus, while 
estimates of the costs of reproducing a full-length high-school 
program, based on our experience, run to more than $60 per 
student, the printing of a larger number of copies would yield a 
considerably lower figure. For instance, if it is assumed that 
printing methods similar to those employed in reproducing col- 
lege workbooks or paper -bound editions of classics are used, 
15,000 frames might be provided in programed textbook form for 
as little as $10 or $12 per copy.* 

It has been suggested that substantial economies can be real- 
ized if the materials are prepared in such a way that they are 
reuseable. It then would be possible to amortize the cost of the 
program over several students as textbooks are today. Tech- 
niques for insuring the reuseability of programed materials have 
not shown much promise, however. One possible solution is to 
have the student make only covert responses rather than having 
them make overt responses that are written into the book. 
Whether or not covert responding is as satisfactory as overt re- 
sponding has not been thoroughly investigated. Until it has been 
satisfactorily demonstrated that covert responding is as effective 
as overt responding, however, it is likely that written responses 
will still be preferable. Another solution that frequently has been 
suggested is to provide the student with an auxiliary answer book 
for recording his responses. As noted earlier, any format that 
leaves previous answers exposed encourages the use of inappro- 
priate cues for responding and should be avoided. Thus, any 
answer record book would have to be designed in such a way as 
to cover previous responses. 

Temporarily setting aside further consideration of programed 
textbooks, it is possible that relatively simple devices can be 
used that present the program on paper disks, paper tape, or 
sheets of paper. A variety of such devices are already available 
at a cost of between $3 and $35. Except for those devices that 
facilitate the reuse of a program, the fact that these mechanical 
methods of presentation depend on printed material makes it 
clear that substantial economies will rarely result, and teaching 
machines of this type cannot be expected to reduce the cost of 
programed instruction materially. Furthermore, when one con- 
siders the cost of the device itself, the expense per student is in- 
creased 



*The programed text, English 2600, written by Joseph C. Blumenthal 
and published by Harcourt, Brace and Co., contains 2,600 frames of the 
constructed-response type. It sells for $2.60. The program by Holland 
and Skinner, The Analysis of Behavior, published by McGraw-Hill Book 
Co. contains somewhat fewer frames. It retails at $5.95 for the cloth 
bound edition, less in paperback. 
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It has been estimated that, with such devices and assuming that 
the mechanism includes an answer tape on which the student can 
record his answers, it would be necessary to furnish a class or 
group with only one -tenth as many programs as there were stu- 
dents. On the other hand, the preparation of such a limited amount 
of materials will increase the cost on a per-copy basis. In addi- 
tion, it currently is not possible to reproduce paper tapes by any 
reasonable printing process; tapes must be printed in sheet form 
and then glued together. While inexpensive devices of this sort 
potentially will find acceptance, it is not likely that the combined 
cost of the materials and the devices could be reduced below $3 
or $4 per student per course per year. 

Turning to a third possibility, the use of microfilm, we find 
that devices are again required and that these devices can range 
from a simple 16 mm films trip projector to highly complex ap- 
paratus such as the Wyckoff Film-Tutor. These complex devices 
are priced as high as several hundred dollars each, but simpler 
film devices could probably be constructed to sell for consider- 
ably under $25 and be useable for about five years at the rate of 
five uses per day. 

Film can be relatively inexpensive when compared with printed 
pages. Once the original is prepared, copies can be made inex- 
pensively. For example, 15,000 frames can be reproduced on less 
than 400 ft of 16 mm film for under $5. This cost almost invari- 
ably would be less than half that involved in preparing a printed, 
programed textbook that includes the same amount of material. 
Furthermore, film generally is more easily stored and consider- 
ably more durable than printed materials. Basically, it also is 
less expensive to prepare. If it could be assumed that each film 
strip would be used for a period of five years, and that it could 
be used by two students concurrently, the net cost would be only 
50 cents per student per course per year far less than the cost 
of presentation by means of programed textbook, or the simple 
devices previously described. 

For many purposes then, it is likely that the presentation of 
auto -instructional materials on film might be the most economi- 
cal procedure. The widespread use of film will depend, however, 
on the availability of a less expensive device than is now obtain- 
able. A device costing only $12.50 still represents an expenditure 
of 50 cents per student per course even when it is assumed that 
the device will be used five times a day and will not require re- 
placement for five years. Although such devices are not currently 
available, it seems clear that they could be produced commer- 
cially if the market were large enough. 

On the basis of a projector that costs perhaps $25, or $1 per 
student per course, plus a cost of 50 cents per student for the use 
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of the filmed program, it is possible to estimate that auto-instruc- 
tional materials presented by means of film would be relatively 
inexpensive. Of course, to this cost must be added the estimate 
of 50 cents per student for the development of the program. But 
even then, it is quite conceivable that auto-instructional programs 
can be utilized for no more than $2 per student per course per 
year when the program has been reproduced on film. Only when 
the volume of sales reached very large numbers would it be fea- 
sible to duplicate these low costs in book format. If an estimate 
is made on the basis of inexpensive, large -volume, pocket-book 
editions, it is quite possible that 15,000 frames could be repro- 
duced for about $4. If multiple uses of each copy is assumed, 
and some solution to the problem of recording answers is 
achieved, the cost per student could be as low as $1.50 for each 
course. Both this estimate and the one indicated for microfilm 
devices are within the range of magnitude considered acceptable 
for a high-school textbook, and are less than the cost of many in- 
dustrial training manuals. 

In addition to these immediate considerations with respect to 
devices, additional economic problems will have to be investi- 
gated before full advantage can be taken of auto-instructional 
techniques in educational, military, or industrial instruction. It 
has been suggested, for instance, that devices are more desirable 
than programed textbooks because they might prevent cheating 
and produce increased motivation. The effects of such device 
characteristics have not yet been demonstrated, however. A 
further problem to be considered before devices are purchased 
is that of the availability of programs prepared so that they fit 
the device. Until the industry promotes some standardization, it 
is unlikely that more than a limited number of programs could 
be used in any one kind of teaching machine. 

The introduction of auto -instructional materials into the class- 
room or industrial training situation also must be considered in 
terms of the merits of this form of instruction. In spite of the 
fact that programs might be more expensive than conventional 
textbooks, it is possible that, dollar for dollar, programed ma- 
terials produced substantially greater achievement. How much 
one is willing to pay for learning is a question that cannot be 
evaluated experimentally. Even if programed instruction were 
only equal to that offered by the average "live" instructor, there 
is reason to believe that auto-instruction can provide more eco- 
nomical education and training than can be achieved by current 
methods. It might be supposed, for example, that the introduc- 
tion of teaching machines might reduce the need for teachers, at 
least to a limited extent. With assistance from programed ma- 
terials, each instructor would be able to supervise and give per- 
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sonal instruction to far greater numbers of students than is possi- 
ble at the present time. 

Suppose, for example, an industrial concern offered a four- 
week training course that involved 3 hr of instruction each week- 
day, or a total of 160 instructional hours. For purposes of this 
analysis, let us assume that the course is offered at a central 
location requiring the students to live away from home for about 
27 days. While he is receiving instruction, the student's morale 
is apt to reflect his separation from his family, his job productiv- 
ity is zero, and the student with greater ability must remain in 
training until the slower students have learned. Without auto- 
instruction, the industry also must pay instructors, furnish class- 
room facilities, be generally content with current levels of 
achievement and with lack of uniform quality from instructor to 
instructor. Ignoring all of these considerations, let us consider 
only the daily allowance of $15.00 provided by the company to off- 
set the trainees living expenses while at the school. This expense 
alone amounts to approximately $400 per student for the four week 
course. If that course were to be programed, the 160 instruc- 
tional hours will represent no more than 16,000 frames. Even 
taking a high estimate of $10.00 per frame including tryout, the 
total cost of preparing the program would be only $160,000. It 
can be seen, then, that the total cost of preparing the program 
could be amortized over only 400 students on the basis of the cost 
of per diem alone. 

In educational situations, of course, students cost nothing but, 
in spite of their low salaries, the overall cost of teachers is still 
considerable. Based on an estimated annual salary of $6 ; 000, a 
teacher who averages 30 students per class is paid approximately 
$200 per pupil per year. The programing of a full year of high 
school would involve the preparations of roughly 75,000 frames. 
At $5.00 per frame, this would amount to an outlay of $375,000, 
which seems to be a large sum but which actually represents the 
average annual expenditure for only 1,875 students far less than 
1% of all students enrolled in any one grade at one time. Even if 
it is assumed that some teachers still will be required, with as- 
sistance from programed materials, each teacher would be able 
to supervise and give personal attention to far greater numbers 
of students than is possible at present, at a far smaller overall 
cost. It is clear then that, compared to the costs of instruction 
at the present time, the seemingly large outlays required to pre- 
pare auto-instructional materials are actually relatively small. 

From the cost estimates that have been described, it is possi- 
ble to draw five conclusions. First, compared with the cost of 
reproduction, developmental costs are relatively small. Second, 
an inexpensive microfilm-presenting device is needed, and when 
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it becomes available, programed materials can be provided to 
students at roughly $2 per course. Third, only when large num- 
bers of copies are involved will it be economical to present auto- 
instructional materials in programed-textbook form. Fourth, the 
apparently substantial cost of preparing a program actually is 
relatively small when compared with the cost of providing instruc- 
tion by more conventional means. And, fifth, although it is likely 
that the cost of auto -instructional materials might not be low 
enough for general use for the next several years, there is every 
reason to believe that economic considerations will not offer last- 
ing obstacles to the wide-scale adoption of auto -instructional 
methods for enhancing student achievement. 



Programed Learning: An Overview 
of Personnel and Financial Requirements 



By WILLIAM DEUTSCH 

Corrigan Communications, Incorporated 



Many preliminary research reports have made such statements 
as "Through programing, we have achieved a 50% increase in the 
effectiveness of our training procedures." There is reason to 
believe that such reports at some levels are reasonably accurate, 
but there are not now available enough original research papers 
to verify this. 

Industrial research is, for the most part, not readily available 
for public distribution. Industrial concerns currently pursuing 
research in automated instruction usually reply to requests for 
information with such generalities as "a 50% reduction in train- 
ing time," "a perceptible gain in training effectiveness," "a 
40% increase in the effectiveness of training." One leading in- 
dustrial representative answered inquiries with "Our research 
thus far has not been controlled to the extent that we would want 
the results reported to .... scientific organizations ...." It is 
hoped that, by next year, well controlled, thoroughly documented 
research reports will be released from some of these industrial 
sources. 

Figure 5-5-1 tends to support this prediction for next year's 
research reports. It serves to emphasize the fact that we are on 
the verge of an industrial expansion that is undoubtedly related 
to our population growth. 

As can be seen from Fig. 5-5-1, the last three years have been 
prolific in terms of instructional devices and programing. In 1959, 
there were only three companies making an industrial effort in 
the auto -instructional field: Rheem Manufacturing, Western De- 
sign, and Corrigan and Associates; in 1960 there was a potential 
market worthy of the investments of 17 major industrial concerns. 
The number of companies has continued to grow until, by June, 
1960, there were better than 63 major industrial corporations an- 
ticipating investments of approximately $1,000,000 each into this 
new industrial market. At the present rate of growth, it is diffi- 
cult to estimate what the situation will be in 1962. As one investor 
stated recently, " Automated instruction represents one of the first 
new major industries in the last fifty years." 

Why has there been this tremendous interest and growth in this 
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Fig. 5-5-1 Number of major industrial companies active in the auto- 
instructional field 

field in the past three years? Psychologists and educators have 
been aware of these techniques for decades at least as long ago 
as 1920 when Dr. Sidney Pressey started some of his early work. 
There are two reasons for this rapid growth at this time: first, 
as mentioned above, the population growth and second, perhaps 
a less direct cause, the result of the publicity resulting from the 
initial research sponsored by the three industrial pioneers men- 
tioned earlier and carried out at various universities and colleges 
throughout the country. 

These early experimental projects lent themselves very well 
to journalistic sensationalism. Such publicity not only left men- 
tal images that sometimes were derogatory but, also, created 
what might be called a "threat -bypass" field. Many parents im- 
mediately hurried to educators at colleges and universities and 
asked such questions as "Will our children be taught by robots?" 

These questions stimulated the thinking of educators who con- 
sulted with industrial concerns throughout the country to ask 
whether it was possible to teach with machines. The industrial- 
ists, in turn, taking it for granted that it was possible to teach 
with automated devices, asked their marketing specialists, "Is 
this a feasible marketing area?" The marketing specialists, 
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finally, tried to view the field from the perspective of financial 
possibilities. 

FACTORS INVOLVED IN CONVERTING TO AUTO-INSTRUCTION 

The first important factor in the automated instruction pro- 
cess is the preparation of the program. This involves variables 
such as the following: 

a. The length of the program, 

b. The type of programing required. 

c. The complexity and organization of the subject matter. 

d. The level of training competency desired. 

e. The predetermined level of student comprehension. 

The type of programing selected will, to a large extent, deter- 
mine the type of machine that can be used. The degree of diffi- 
culty of subject matter has the effects shown in Table 5-5-1. 

Table 5-5-1 Programing Conversion Data 



Subject- 


Length of 
conventional 


Length of 
programed 


Frames 


Preparation time 






matter 


course 


course 


prepared 


Man- 


Man- 


level 


(hours) 


(hours) 


per hour 


hours 


months 


Difficult 


20 


11.66 


1.5 


699 


4.13 




10 


5.83 


1.5 


349 


2.07 




5 


2.92 


1.5 


175 


1.04 


Medium 


20 


11.66 


2.0 


525 


3.10 




10 


5.83 


2.0 


262 


1.55 




5 


2.92 


2.0 


131 


0.77 


Easy 


20 


11.66 


2.5 


420 


2.48 




10 


5.83 


2.5 


210 


1.24 




5 


2.92 


2.5 


105 


0.62 



These data are derived from the savings in training time re- 
ported for a number of industrial experimental research pro- 
jects. The savings varied from 33% to about 50% over conven- 
tional training courses. The average was about 41%. Thus, we 
can see that a conventional training course with material of 
medium difficulty requiring 10 hr of training time would re- 
quire only about 5.83 hr of programed or auto -instructional 
training a saving in training time of approximately 41% . 

The number of frames prepared per hour represents the 
average number of frames we can expect an experienced pro- 
gramer to write in an hour. The sixth column shows the total 
man-hours required for a programer to produce this material, 
and the seventh, the total number of months required for the 
preparation of the program. 
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FACTORS INVOLVED IN PRODUCING PROGRAMS 

Why is it that a good programer can only produce 1.5-2.5 
frames per hour? What does he do? In attempting to answer 
these questions, we can divide his tasks into seven phases as 
follows: 

1. The preparation phase in which he defines his behavioral 
objectives, defines the population, and outlines the material in 
sequential form. (This requires about 25% of the total program- 
ing time and costs.) 

2. The completion of the first draft of the programed seg- 
ment. (This consumes about 30% of the time and cost of pro- 
graming.) 

3. Testing the original draft, usually with a few student sub- 
jects, which testing results in subsequent revisions of the ma- 
jor problem areas. (This takes about 15% of the total program- 
ing investment.) 

4. Testing and rewriting the revised draft using a larger num- 
ber of student subjects. (A cost of about 10%.) 

5. Testing and revision of the third draft. (This effort repre- 
sents 5% of the total cost.) 

6. Final program validation. (About 5% of the investment.) 

7. Final editorial work. (This might cost about 10%.) 
Many industrial concerns are moving rapidly from the re- 
search phase of automated instruction into the area of direct 
application. Leaders and representatives of universities, in- 
dustry and the military are carefully evaluating their specific 
training needs to determine whether or not they are ready for 
automated instruction. Perhaps at some time during the coming 
year more adequate information will be made available to assist 
them in making decisions of this type. Until that time, however, 
they must decide on the basis of information now available. 



Programed Instruction: Some Economic 
Considerations 



By STUART MARGULIES 
Center for Programed Instruction, Incorporated 



Many training directors have become convinced of the advan- 
tages of programed instruction. Decreased training time and in- 
creased efficiency are improvements that every company can use. 
Industrial training is complex, however, and it is difficult to de- 
cide where programed instruction should be introduced into the 
industrial -training system. The present article attempts to de- 
fine the desirable characteristics of an initial course of programed 
instruction and provides some estimates of the costs involved. 

AREAS OF APPLICATION 

Programed instruction is a method of teaching; hence, in 
theory, it should be equally applicable to any body of knowledge. 
As the Mechner and Cook article (see section 1) suggests, how- 
ever, subject-matter material must be broken down into small 
bits of information, and this information must be sequenced to 
lead cumulatively to increasingly more complex material. In 
the academic world, some subjects, such as mathematics and 
the natural sciences, naturally meet this requirement for auto- 
mization. Others, such as theory of education and poetry ap- 
preciation, do not. Similarly, in industry, some subject-matter 
areas are more readily structured in a step-by-step fashion 
than are others. 

Several of the articles in this volume, notably the staff paper 
by Basic Systems, Inc. (see section 5) and the several case 
histories, all suggest industrial applications that are best suited 
to the "sequential" approach. Some of the successful step-by- 
step industrial programs are used at present to teach such sub- 
jects as knowledge of pharmaceutical products, computer- 
system interrelations, billing-clerk procedures, computer 
language, endocrinology, and (the most common industrial pro- 
gram) basic electricity. 

Let us assume that we are called in by an insurance company 
to determine which courses should be programed first. On the 
basis of the considerations outlined above, our first approach 
would be to select those courses best meeting the requirement 
for step-by-step development. We would select subject-areas 
having a large amount of factual material. In addition, for 
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practical purposes, the courses selected would be stable mem- 
bers of the training curriculum, rather than ones that might be 
obsolete in a year or two. The courses should also be of suffi- 
cient importance to the overall training program to warrant the 
expense of programing. 

PRIORITY OF APPLICATION 

Continuing our analysis of the insurance company's training 
program, we find that they have a large variety of factual 
courses. In deciding which of these to program first, the follow- 
ing considerations would govern our choice; 

Validity criteria. Management will assuredly demand an eval- 
uation of the new training procedure. In many cases, however, 
it will not be possible to compare the new programed-instruction 
course with a conventional learning situation because no eval- 
uative criteria have been developed for the conventional courses. 
It is important, therefore, to find those existing courses that do 
have some validating criteria. Measures such as final examina- 
tion scores, number of work-items completed, or errors made 
on the job will be acceptable. Examinations prepared outside 
of the company are particularly desirable because they have a 
high acceptance value to management. For example, we might 
select a course that is followed by a State licensing examination. 
This will enable management to compare immediately conven- 
tional training results with those obtained with the new pro- 
gramed-instruction technique. 

Length of course. Preparation time for programed materials 
can be quite extensive (estimates are given later) . Lengthy 
courses should not be undertaken until success is achieved with 
shorter courses. Initial success with a short course will serve 
to reassure an uncertain management and to provide important 
feedback for longer ventures. A programed course replacing 
25 hr of conventional instruction is ideal. Longer courses, on 
the order of one month of conventional instruction (160 training 
hours), should not be undertaken in the beginning. 

Techniques of integration. The Hickey article (see section 7) 
provides many important insights into the problem of integrat- 
ing the new training procedure into the existing training sys- 
tem. Students taking programed instruction will complete 
course work at different times and therefore, will not be able, 
to move as a group into a later phase of the conventional train- 
ing situation. For example, if our insurance company presently 
has a 60-hr training course, it would be unwise to select the 
middle 20 hr of this course for programing. Because the stu- 
dents will finish at different times, continuation of the conven- 
tional training course will present significant problems, and 
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using the new technique might be more disruptive than other- 
wise. On the other hand, if a terminal portion of the conven- 
tional classroom course is converted to programed form, and 
is then followed by individual field training, a more useful ap- 
plication will result. 

Another consideration that will affect the integration of the 
programed -instruction course into the training system is the 
efficiency of the existing conventional courses. We frequently 
find that more-or-less untrained, part-time lecturers are used 
for certain course material, and the resulting efficiency of 
training is low. A particular attempt should be made to use 
programed instruction for these courses. 

Rapid turnover. Programs can be very costly (estimates are 
given later) and, hence, should be rationalized over a large 
number of students whenever possible. For this reason, and 
for convenience, most existing programed industrial courses 
have been developed at the introductory level. Notable excep- 
tions are those special courses that were developed for highly- 
paid personnel, typically in the upper levels of management. 
Training time in these instances is usually so costly as to 
warrant the introduction of programed material at an advanced- 
course level. 

On-the-job training. The potential reduction in cost possible 
by using programed instruction for on-the-job training is an 
obvious advantage. In many industrial and military situations, 
a large part of the cost of conventional training results from 
travel expenses and per-diem payments incurred in transport- 
ing field personnel to centralized locations for training. Courses 
taught through programed instruction provide an obvious solu- 
tion to this economic problem. In addition, present indications 
suggest that field sales representatives are highly motivated 
to learn from programed-instruction materials in their own 
time after the normal working day. Thus, sales activity is not 
interrupted by the time required for training. 

LENGTH OF PROGRAM 

It is difficult to estimate the number of frames required to 
translate a conventional training course into programed-instruc- 
tion form. The measure of program length, however, is of basic 
importance in determining the preparation time (and cost) of the 
program. Estimation difficulties arise because programs can 
not be written by a simple ''translation'' of the conventional 
learning material to programed form. Instead, the conversion 
from conventional text to programed frames must be preceded 
by an often lengthy task analysis, in the course of which impor- 
tant curriculum revisions might become evident. 
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A good estimate of program length can be made easily if the 
programed material merely replaces formal company lectures. 
Several industrial studies have indicated that about 50 programed 
frames are needed to replace each hour of classroom time. This 
applies to representative courses, such as basic electricity or 
computer-system familiarization, but does not apply to curricula 
containing high computational components. As an example of the 
latter kind, a 1-hr course in payroll disbursing procedures 
(which requires the solution of lengthy problems) can be re- 
placed by only 20 computational-type frames. 

Although about 50 frames will provide the same material or- 
dinarily taught in a 1-hr classroom lecture, students might be 
able to complete the programed material in much less than 1 hr, 
with a resulting decrease in training time. Several studies in 
secondary schools and colleges have indicated that students can 
finish 90-100 frames per hour, yielding a 50% reduction in 
training time, but industrial training applications have, more 
typically, yielded reductions of 25 to 30% . 

PROGRAM PREPARATION TIME 

An experienced programer, with the aid of staff specialists 
(supervisors, editors, draftsmen, etc.) can produce, typically, 
about 100 tested and revised frames per week. Including the 
time required for a preliminary task analysis, a program of 
2,000 frames will require, therefore, about six man-months to 
complete. Additional time will be required for new, highly com- 
plex material; less time will be required for material that is 
similar to courses previously prepared. 

PROGRAM COST 

Programing companies typically charge a single fee for their 
services, which includes both the task analysis and the prepara- 
tion of programed course material. Thus, the fee usually is 
quoted on a total cost basis rather than on a cost-per-frame 
basis. Nevertheless, it is clear that the number of frames is 
a major determinant of cost, and this number may be used as 
a basic reference unit for cost comparison. Accordingly, the 
cost analysis presented here is made on a cost-per-frame basis. 

In the discussion of costs, program length will be set, arbi- 
trarily, at about 1500 frames. This assumption of a 1,500-frame 
length is important because the cost of training analysis is 
proportionately greater for short programs than for long ones. 
In addition, other startup costs for both short and long programs 
are relatively fixed so that programs of greater length cost 
somewhat less per frame than shorter programs. 

Other considerations affecting per-frame costs are the tech- 
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nical complexity of the course material and the availability of 
trained personnel. As the course material becomes increasingly 
more technical, the availability and training of programing per- 
sonnel grows increasingly more critical and costs go up. Mili- 
tary material that is classified as "Secret" or higher is very 
difficult to handle, and greatly inflates all cost estimates. 

Several other articles presented in this book, notably those 
by Hickey (section 7) and Ross (section 5), provide good exam- 
ples of cost estimates. The cost figures given below are indica- 
tive of the currently prevailing prices. They have been obtained 
from polling numerous programing companies, from an analysis 
of representative bids made on military contracts, and from a 
review of present industrial contracts. These sources yield 
cost figures that are slightly higher than those obtained by 
Hickey and Ross, i.e., a cost range between 8 and 30 dollars 
per frame with median values between 14 and 20 dollars. These 
costs reflect the price for a " tailor-made' ' program that is 
suited to a particular industrial training requirement and that 
remains the exclusive property of the contracting company. 

Proprietary rights are an important cost consideration. Pro- 
graming companies are presently charging a premium of 25- 
150% of the basic cost of the program when all rights are the 
property of the contracting company. Although this premium 
estimate probably will remain reasonably valid for the bulk of 
industrial training programs, proprietary programs in basic 
electricity, and other courses in great demand, soon might be 
marketed at much less than their "tailor-made" price. 



SECTION 6. Machines and Devices 



To many people, the terms "teaching machine" and "pro- 
gramed instruction" are synonomous; the teaching machine, how- 
ever, merely acts as a vehicle for presenting a program. In many 
cases, a machine is a necessary component of a teaching system, 
but in other cases, the machine can be replaced by a special type 
of book called a programed textbook. 

Although several experiments have shown that programs teach- 
ing verbal skills can be presented with equal efficacy by teaching 
machines and programed textbooks, the nature of the training task 
often requires some sort of machine. In any event, there is justi- 
fication for a teaching machine if we want to change the behavior 
of the learner more precisely than we can with a programed text- 
book. There also is a long-run economic advantage in the use of 
certain types of teaching machines in that the programs can be 
produced on media other than printed paper. These media will 
very often serve many more students than the usual type of book. 

The articles in this section deal with the subject of teaching 
machines without reference to presentation by programed text- 
books. The editors have attempted to collect articles that cover 
various aspects of teaching machine specifications, usage, and 
design. 

The first article, by Foltz, is an overview describing many of 
the existing machines, both commercial and noncommercial de- 
vices. The reader will find a reasonably extensive list of devices 
and their descriptions. Because the field is changing so rapidly, 
many of the machines mentioned in this article will be obsolete 
or out of production shortly, and new devices will be available by 
the time this volume has been published. Specifications and prices 
will also change with improved engineering techniques and changes 
in the market. 

Because there are so many companies involved in the manufac- 
ture and design of teaching devices, it is impossible to assume 
that the list given in this article is complete. There are undoubt- 
edly companies and devices that are not represented in the arti- 
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cle. The editors, therefore, recommend that the reader regard 
this article as a representative sample of current practices 
rather than a complete list. 

The second article, by Eigen, describes the mechanics of edu- 
cation systems that, strictly speaking, would not be classified as 
implementations of programed instruction but that share many 
common characteristics with it. The reader interested in modifi- 
cations of programed instruction for specialized training tech- 
niques will find many examples of current practice outlined in this 
article. As is the case with Foltz's article, "Pseudo-Programed 
Instruction" should not be considered an exhaustive compendium 
of current practice; rather, it is a list of some of the more inter- 
esting applications of several of the elements of programed in- 
struction to specific types of training tasks. 

The third article, by Silvern, describes a continuum of develop- 
ments dealing with electronic -systems maintainence as it is in- 
fluenced by performance requirements, studies of behavior pat- 
terns, technical-training requirements, and engineering-design 
and management philosophies. The editors have selected this 
article as an excellent example of the teaching machine in the 
context of a training problem. Although there have been many 
articles written on teaching machines, this particular paper will 
give the reader some insight into many of the types of decisions 
that will be required in the creation of a training system. The 
majority of the procedures described in this article, in the opinion 
of the editors, can be generalized to most training problems, and 
hence it is recommended that the reader regard this paper as a 
reasonable working model from which he can investigate the ap- 
plication of teaching -machine technology to many other training 
tasks. 

The final article in this section, by Levine and Knight, briefly 
describes two types of teaching machines and deals with some of 
their advantages and disadvantages. This article will prove in- 
valuable to anyone considering long-range plans to install equip- 
ment for teaching purposes. 



Aids to Teaching: A Survey of the 
Current Status of Teaching Machines 

By CHARLES I. FOLTZ 
Electronic Teaching Laboratories, Incorporated 



Technological aids to teaching can be classified in three cate- 
gories. In the. first are the stimulus devices, which include most 
audio-visual media. These assure the broadcast of material con- 
taining educational content, but there is no assurance that the 
educational processes will be carried out by the student. In the 
second, response jdevices, such as the- typewriter, give the stu- 
dents a method of practicing and drilling responses but do not 
supply educational content. Teaching machines fall into the stim- 
ulus-response-device or third category, and add to the first two 
the concept of immediate reinforcement. How much this aids the 
student in his learning process is something that is now subject 
to intensive studies of various sorts. 

Existing results seem to indicate that teaching machines, pro- 
perly employed and programed, might considerably increase 
learning speed, reduce the age level at which advanced content can 
be presented (the Skinner method shows students complex con- 
cepts before they know what they are, and then they are slowly 
taught by "augmenting' J to comprehend them), and provide a new 
type of motivation and/or practice for the student. Results also 
indicate, claim the enthusiasts, that automated instruction can 
compete fairly successfully with other educational methods (and, 
even, outside interests) for student attention. Some people in the 
field, however, believe that there has not been enough experimen- 
tal control in existing research projects to show that teaching by 
an automated method is any better than the conventional method 
of teaching. 

Two main advantages of teaching machines are their versatility 
and their infinite (machine) patience. Programing and designing 
devices for teaching industrial skills is a specialized field and 
an exciting market. Several firms are now engaged in testing 
audio-visual devices with film and synchronized tape programs 
to teach assembly processes. The results so far have been ex- 
cellent. Programs and devices to teach incremental motor skills 
of this sort would be relatively easy to construct because most 
of the information and all of the technical knowledge already 
exists. This might be an excellent field to study some of the epis- 
temological questions presented by machine teaching in view of 
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the fact that the investment capital necessary for development is 
more readily available here than in the education field. Several 
devices and programs have been tested with preschool and "ex- 
ceptional" children and have a fairly high record of success. 
This special problem area is one that is singularly adapted to the 
use of the automated tutoring method, and, again, a large-scale 
market (parents and institutions) seems to be a fairly immediate 
possibility. 

QUESTIONS THAT NEED ANSWERING 

Comparative studies of the different methods of programing 
and their applications to various disciplines are needed to make 
evaluations. Considering the instructional goals, what disciplines 
should be taught by automated teaching methods? Is it always 
better to minimize the number of errors in the learning process? 
How should the program be organized? What is the form that the 
response elicited should take? If cueing and prompting are to be 
used, what form should they take? What is the best step size 
considering the age level of the students and the discipline being 
taught? What form should feedback take: that of giving the right 
answer immediately after the student makes a response, merely 
informing the student that he is wrong and should try again, giving 
the student reasons why he is wrong and sending him back to 
choose a right answer, or giving the reasons followed by the cor- 
rect response? What is the educational value of minimizing aver- 
sive reinforcement? What is lost and/or gained? 

Once most of the above questions are answered, the device is 
almost designed. Economic questions now come into play and one 
must decide whether automatic scoring provides enough educa- 
tional benefit to compensate for the increased cost of the device, 
what is the best form of presenting the program (film, paper pro- 
gram, etc.), and whether the device should be adapted to various 
modes of operation. 

Certain over -all theoretical questions must be asked before 
the value of machine learning can be ascertained and before this 
method can be used to any great extent. What are the advantages 
of active participation, particularly in relation to student age 
level and I.Q.? (Dr. Lumsdaine seems to believe that they will 
decrease as the I.Q. increases.) How much long-term retention 
and transfer of knowledge learned results from the use of the 
automated teaching methods, and how does this compare to the 
present methods of teaching? What function should the teaching 
device have in the overall educational program? (Should it be 
used to supplant human teaching of certain disciplines?) What is 
the place of the teacher in the educational program if automated 
instruction is to be used to any great extent? (Pressey believes 
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that teaching devices should be used only for classroom prepara- 
tion, i.e., more or less automated homework machines. Others 
believe that all materials should be presented in the programs 
and/or devices. These same people cloud the issue of the teacher's 
functions in a nebulous haze of generalities; they say a teacher ' 
will still "inspire the answers ," teach the students to think and 
write well, give counselling, and stimulate the student to original 
thinking and work. What does this mean?) 

THE EXISTING DEVICES 

As in any field in which rapid expansion has been brought about 
by the introduction and marketing of potentially profitable new 
products (Electronic Industries enthusiastically estimates a 
100, 000, 000 -dollar/year market for the next 10 yr), development 
of automated teaching devices (and, particularly, of concomitant 
program material) has been uneven. Many firms rushed to get 
into the field with a machine, any machine, forgetting almost en- 
tirely about programing and, even, the fundamental educational 
purposes for which the machines were intended. 

As of this writing, there are 14 companies that produce various 
types of automated teaching devices and eleven that have announced 
that they are doing research with the intent of entering the field. 
There are, in addition, some 22 noncommercial devices that have 
been constructed and tested in classroom or laboratory work. 
Both the commercially available and noncommercial devices are 
listed, separately, in Tables 6-1-1 and 6-1-2. 

A teaching machine should consist of five elements: a data 
storage receptacle that will contain the program to be presented 
to the student, a display mechanism, a response panel for either 
a multiple choice selection or a constructed response, some sort 
of feedback mechanism by which the student can get immediate 
knowledge of the correctness of his response, some sort of rein- 
forcement or reward mechanism, if necessary (this may be com- 
bined with the feedback mechanism). 

Teaching machines may be roughly divided into two categories, 
depending on the type of response (constructed or multiple choice) 
the student is required to make. They can be further subdivided 
into degrees of complexity, which include considerations of the 
form in which stimuli are presented, the method of presentation, 
and the semi-to-complete automation of scoring, operation, and 
feedback. 

Constructed-Response Devices 

In the constructed-response categories, the simplest device is 
one designed by Porter (see Tables 6-1-1); it is completely manu- 
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ally operated. The data are scored on a single slip of paper, with 
five responses required, which are to be written on the program 
itself. Feedback is provided by moving the answer up under a 
glass shield ("cheatproof") which also brings the correct answer 
into view on the display slot. The disadvantages of this arrange- 
ment are that the students can look at the program and answers 
before inserting them into the machine, and, because the students 
write on the program itself, it may not be re-used. A further dis- 
advantage is that the student is required to score himself so that, 
in actuality, the device is by no means "cheatproof." 

Slightly more sophisticated devices of this type have been de- 
signed by Skinner and others. They are also manually operated, 
but the commercial devices can hold about 1/2 - 1 hr of pro- 
gramed material, which is locked into the machine so that the 
student cannot "peek" at the answers beforehand. They are still 
not "cheatproof", despite such items as a "non-cheat spring" 
(which prevents students from moving the response tape in order 
to rewrite their answer), because they are student scored. Most 
of these devices can, at some extra expense, be made completely 
automatic in operation, but, because the basic drawbacks and 
mode of operation still remain, this would be an undesirable ex- 
pense. 

The manually-operated Skinner device (Dynaslide) to teach 
basic arithmetic (see Table 6-1-2), and a similar, commercially 
produced, device that is electrically operated (Smith -Harrison 
Model 80), add new features to the basic device structure. Simi- 
lar to the aforementioned devices in other respects, they are so 
geared that when a wrong response is made, it is impossible to 
move on to a new problem. In addition, each incorrect response 
is automatically scored (on a dial on the side of the Skinner de- 
vice and on a separate strip of paper in the other). 

Gilbert has suggested a modification of this basic Skinnerian 
device. He would have a program on paper presented through a 
display window, and the student would answer with a constructed 
response written on a separate piece of tape. The student operates 
a device to open another window that displays 12 possible answers. 
He then dials the number of the answer (on a device supplied at 
the side of the machine) that most closely corresponds to his own 
answer. This drops a new frame into the display window. Thus, 
this device provides for a branching program that is dependent 
on the student's own response. 

Rothkopf's Autograph is less automated, but it introduces a new 
mode of stimulus display. The stimuli are presented on slides on 
a screen in front of the student, and, when the student has com- 
pleted writing in his answer on a separate sheet of paper in the 
device, he rolls it up under a glass plate that, at the same time, 
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removes a cover over a section of the slide to reveal the correct 
response. The student then scores himself. The display is regu- 
lated by an instructor who is hidden from the student and who can 
change the program sequence according to the information he re- 
ceives on the student's accuracy of response. This leads us into 
various other, more complex, intrinsic programing devices. 

Roth, Anderson, Perli, and others propose highly complex ma- 
chines using an automated ' 'little -man-in~the -box' ' technique. The) 
display can take the form of either a written program or, more likely, 
a filmed program. Responses are made on a typewriter keyboard 
and the answers fed into a computer that analyzes the answers 
character by character. The computer then informs the student 
of the correctness of his response and analyzes instantaneously 
the student's problems and needs, and then programs accordingly. 
Such a system would be tremendously expensive, but the expense 
would be somewhat reduced by having a single computer handle a 
number of student positions simultaneously. Such a device is cur- 
rently only in the research and development stage and the enorm- 
ous problems of programing the computer to handle the system 
have not yet been precisely worked out. 

Multiple -Choice -Response Devices 

The range of devices in the multiple-choice-response category 
is even more diverse because of the greater technical ease of 
checking multiple -choice answers. The simplest devices are 
manually operated, student scored, and extremely simple in op- 
eration. Under this category, comes Pressey's punchboard device. 
The student puts a question sheet over a perforated block and 
punches a hole through the paper next to the answer of his choice. 
If the pencil penetrates, his answer is correct. The Chemo-Card 
also comes under this category. With the Chemo-Card, the stu- 
dent marks his choice with a damp stylus. The paper is so treated 
that the mark turns a certain color if correct and another color 
if incorrect. 

Pressey's Memory Drum Device has the additional features of 
machine scoring (number of incorrect responses) and provision 
for preventing the student from continuing the program until he 
has made the correct response. When a question is answered 
correctly, it will be dropped out of the sequence. Similar in prin- 
ciple, although somewhat more complex, are the Army's Auto- 
Rater, the Navy's Self -Rater, and the Green Light Rater, as well 
as Brigg's Card Sort Device. These differ in data-storage ca- 
pacity, the type of display, and the method of feedback. Informa- 
tion as to the correctness of answers is provided by a green light 
for a correct answer and a red light for an incorrect answer. 



232 MACHINES AND DEVICES 

These lights are geometrical in shape, such as a circle and a 
triangle, to cope with the problem of color blindness. 

Much more complicated devices have been designed by Crow- 
der that operate on intrinsic program principles. The Tutors, of 
Crowder design, present data on film. The student sits at the 
console and can dial motion or still sequences. According to his 
response, he is told to select a certain frame, which he then dials 
and continues with the program. The data-storage capacity on 
these two devices is 10,000 and 5,000 frames of 35 mm film, re- 
spectively; this would be sufficient for many hours of student op- 
eration. 

Another Rothkopf device, the "Polymath" utilizes a paper-pro- 
gram display device. This device also can have a human opera- 
tor or a computer to correct answers and provide feedback to the 
students. The computer or human operator chooses the slides to 
present to the student on the basis of the student's response and 
provides feedback by means of vari-colored lights that indicate 
the validity of his response. The unusual thing about this device 
is that it has 24 multiple -choice buttons arranged in an open 
square around a constructed-response panel. On this panel can 
be placed a map outline or some similar material that the stu- 
dent is required to trace with an electric stylus. If the tracing 
is correct, the electric stylus completes a circuit, and the stu- 
dent gets immediate knowledge of the correctness of his response. 

Devices for Preschool and "Exceptional" Children 

In a somewhat different category are several devices that are 
designed to teach preschool or "exceptional" children concept 
discrimination. These have a display panel and only two or three 
choices of response. The Hively machine has a large display win- 
dow in which the program appears. Below it are two smaller dis- 
play windows containing stimuli items to be compared or con- 
trasted. To make a response, the two lower display windows 
themselves are pressed. The other devices have similar displays 
but separate response panels consisting of multiple -choice but- 
tons. The student is asked to match or discriminate between the 
pictures presented in the two windows below the display window 
from which the image is presented. In doing this, they learn var- 
ious concepts such as the concept of shape (triangles, etc.). With 
proper programing, this machine could also teach reading. First, 
the written word could be presented in the display window and 
similar words given in the lower windows. The child could be 
asked to match these. Then a picture of the object (e.g., "cat") 
could be presented in one of the lower two windows with different 
words appearing in the other and one of them identifying the ob- 
ject. Research is being performed on programing devices such 
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as these by the Devereaux schools for exceptional children. A 
Yale sociologist named Omar Khayam Moore is doing research 
in teaching preschool children how to read and write. He uses a 
screen and a slide projector to project the images and has the 
child reproduce them. As the program increases in complexity, 
the child is required to identify objects and write down the words 
that correspond to them. 

Another interesting device developed by Dr. Moore is one to 
teach preschool children to type and take dictation. To do this he 
paints the finger nails of each child in individual colors. Then the 
keys that are to be used by each finger are painted in colors to 
match. Drills and programs are presented either on paper or by 
a slide projector. The typewriter is so modified that, if the child 
makes an incorrect selection of a typewriter key, the key will not 
depress. Dr. Moore hopes to connect his system to a computer 
that will program automatically for each individual child. He has 
had singular success with his devices, although his experimental 
group was quite small. He was successful in teaching a three - 
year-old child to print after 14 weeks, and he has a seven-year- 
old child typing and taking dictation using simple vocabulary. 

The two most interesting facets of Dr. Moore's work are the 
age at which a child can be taught to read and write, and also the 
fact that, for machine teaching at least, it is much more simple 
to teach printing and typing than it is to teach handwriting. This 
suggests a revolutionary concept. That is, that handwriting might 
be outmoded as a writing technique. In the interests of facility of 
learning and legibility, it would be much simpler to teach a child 
to print and, perhaps, to type. 

Devices for Industry 

Then there are devices designed for industrial use that are 
mainly for the teaching motor skills, some coupled intellectual 
skills, and drill in both of these. The simplest is the manually 
operated, student-scored Tab Item device for training in elec- 
tronic trouble -shooting. The student is given a symptom of what 
is wrong with the machine he is being trained to repair. He then 
must go through a written series of possible operations and checks 
he can perform and select the ones he thinks most relevant to the 
problem. At the end of each check is a slot covered by a tab. If 
he pulls the tab, he will be given the result of making such a 
check. The aim is to get through the program with the minimum 
number of tabs being pulled necessary to the logical diagnostic 
pattern used in repairing the machine. 

Another company is marketing a modified film projector, and 
a number of programs related to industry on film. This device 
utilizes lecture and example techniques, and infrequent tests are 
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also projected. The tests are answered by the students on sepa- 
rate answer sheets that are provided. This is a student scored 
device; the film gives the answers immediately after each test. 

Hughes Aircraft and Applied Communications Research Sys- 
tems have designed training devices for assembly processes. 
These consist of televised film displays and synchronized audio 
material (which each operator listens to through earphones). The 
operator goes through the described procedures as he watches 
and listens to them. The Hughes system is currently in use and, 
it is claimed, has yielded a great increase in efficiency, an even 
greater decrease in defective assemblies, and an improvement 
in morale (which is not entirely due to the "appropriate indus- 
trial background music' 5 that comes through the earphones when 
instructions are not being given). 

Finally, there is a separate group of devices designed by Gor- 
don Pask of Solartron Electronics, Ltd. These are intended to 
teach the use of the IBM key punch (Hollerith), the twelve -key 
adding machine, and to practice and teach typing. All these de- 
vices use computers to do the programing. They are based on 
the Rothkopf "little-man~in-the-box" technique. Essentially, the 
trainee plays a game with the machine. The machine paces the 
student, increases the speed of stimuli presentation, and the dif- 
ficulty of exercise, always staying a little bit ahead of him. Pask 
reports that the motivation to learn in this manner is tremendous, 
and that trainees have stayed at these devices long after the train- 
ing time is over. The trainees report that they enjoyed working 
with the devices but that they often worked so long that they were 
"bone weary" when they stopped. Such devices adjust to the in- 
dividual problems and idiosyncracies of each trainee individually 
and are the closest approximation to a personal teacher -student 
relationship that has yet been devised in the automated field. The 
reinforcement mechanism consists merely of the challenge of 
playing the game with the machine. There is no feedback except 
in the form of recognizably simpler or more complicated mate- 
rial and accomplishment. 

The Language Laboratory 

The language laboratory as a teaching machine is a special 
problem intrinsic to the instruction of all oral language. All 
stimuli, responses, and consecutive feedback or reinforcement 
require a special type of programing. A straight-line program 
is necessary, and the device must be student scored and cor- 
rected. To correct each student individually would involve a tre- 
mendous amount of time, and, at the present time, the most so- 
phisticated and automated scoring device is a machine that has 
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a vocabulary of only 15 words. To aid student scoring, a device 
has been suggested by which the student introduces a high -fre- 
quency tone into the tape wherever he has made an error. He 
then plays back the tape at a high speed, and when the tone is hit, 
the machine slows down to a normal talking speed. 

Rand Morton at Michigan University is working on an advanced 
language laboratory. His system will consist of a very large lan- 
guage laboratory with a dial system in each student booth. The 
student can dial any one of a number of programs in different 
languages at different levels that are being continually broad- 
cast. Sooner or later, he hopes to have some sort of synchro- 
nized visual device that can teach the written language in con- 
junction with the spoken one. The student will be presented with 
the written word on a screen, it will be repeated several times, 
and he will be asked to reproduce it on some sort of constructed- 
response mechanism on the desk in front of him. Eventually, this 
might be used to teach the student the grammatical rules and con- 
structions of the language as well. 

The Programed Textbook 

The simplest teaching machine of all is the programed text- 
book. There are two types of such textbooks: one requiring a 
constructed response and the other one requiring a multiple - 
choice response that leads into an intrinsic program. Books of 
the former type have been designed by Homme and Glaser and 
the latter, introduced by Crowder, is designated a "scrambled 
book". Not being automated, there is no method of making such 
material either "cheat proof" or scored in any way other than 
by the student himself. 

Several tests have been made that compare the merits of pro- 
gramed textbooks and teaching devices employing programs. The 
studies that have been made indicate that students learn equally 
well from a programed textbook as they do using a teaching de- 
vice. This raises the question of whether student scoring is any 
better than automatic scoring by the machine. Many devices have 
sacrificed automatic scoring mechanisms in the interest of econ- 
omy, but economic considerations are not the only ones. The less 
opportunity the student has to circumvent the learning process 
by cheating, the more efficient the learning process becomes. 
The use of a machine requires the student to follow the prescribed 
order. If he uses a programed textbook he might be able to take 
the easier items first and to look over the whole program and de- 
cide what he wants to complete and what he does not want to com- 
plete. This procedure would negate some of the benefits to be 
derived by this method of teaching. Furthermore, some sort of 
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overt response is required from the student. He must either con- 
struct a response or select one but certainly not copy one, which 
would be possible if he were using a programed textbook. 

In spite of initial appearances to the contrary, the teaching -de- 
vice enthusiasts argue with some logic that it very well could be 
more economical to use a device than to use a programed text- 
book. The argument goes something like this. After the initial 
outlay for the purchase of devices, the maintenance should be 
minimal. Programs can be used and reused by a great number 
of students in most of the teaching devices that now exist. The 
programed textbook, however, will contain some additional mate- 
rial that provides feedback to the student, and might, therefore, 
be more expensive to produce than the machine program. In 
addition, most textbook programs cannot be used over again be- 
cause the student is required to make overt responses on the 
pages of the textbook.* 

Device enthusiasts thus argue that teaching machine costs 
might be made up, in a reasonable amount of time, in savings 
from the decreased expense of buying programs. The greater 
cost of programs, and the greater number of programed textbooks 
required from this point on, make the programed textbook more 
costly than employing any but the most expensive devices, so they 
say. 

SELECTING THE DEVICE 

Each of the component parts of a proposed teaching machine 
must be studied in relation to the program and, particularly, to 
the disciplines to be taught before the device is constructed. As 
far as data storage is concerned, the devices that employ tele- 
vision screens certainly need none. Film and slide devices are 
more expensive to produce, but the film or slide is more durable 
than a paper program and offers more flexibility in display. 

Data-storage capacity depends on whether films or paper pro- 
grams are used. Ideally, a device should contain enough program 
material so that the teacher or, particularly, the student would 
not have to be bothered by having to make frequent changes that 
would interrupt the learning process. A small data-storage ca- 
pacity is the main fault of the Porter device and the Pressey 
punchboard devices and constitutes a lesser drawback in the 
Skinner disc device and the Pressey memory drum. 

The display should be centrally located, in easy view, and have 



* Editor's note: Some publishers, aware of this problem, are providing 
separate answer booklets to partially overcome this difficulty. Books, 
however, are comparatively limited in their durability when compared 
with film or other media. 
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some logical connection with the response panel (e.g., the display 
on the Pask device to teach the use of a twelve -key adding ma- 
chine resembles the adding machine keyboard). Up to a certain 
point, the larger the display the better. The mechanism of the 
display depends entirely on the material on which the program is 
presented. Film, although expensive to produce and display, prob- 
ably provides the best method of displaying programed material 
because of the greater number of items that can be presented on 
each section, the flexibility of being able to display either single 
frame or motion picture sequences, a greater durability, and the 
enhanced attractiveness of the display. 

The response panel should be designed to be easily accessible 
to the student and should be of simple enough construction so that 
the student can master it in a short period of time and be able to 
concentrate his attention on the display. If it is too complicated, 
as in the Polymath device, the student will spend time making his 
selection that he should be employing to study the program. The 
Tab Item device, of course, is merely composed of a response 
panel that actually presents the skill to be drilled and, so, is an 
exception to this rule. 

There have been no studies of multiple -choice -response devices 
that indicate the optimum number of responses that should be 
made available to the student. Probably three or four would be 
sufficient variety without being overly complicated if the device 
were so designed to continually change the response pattern. 
Otherwise, the student would continually press the same button 
or figure out the response series without bothering to learn any- 
thing at all. 

In the constructed-response devices, the responses should be 
made (and generally are) on a separate strip of paper. This al- 
lows the program, if it is a paper program, to be reused and 
proves a type of detachable scoring record for each student. If 
the response is to be constructed on a typewriter keyboard, or in 
some other manner, the same considerations should apply as 
those for the multiple -choice -response devices. 

At this point it is necessary to digress a bit and discuss the 
various modes in which a device can be used. The Subject Matter 
Trainer, despite other deficiencies, was useful in making a study 
in how best to use a teaching machine. Groups of students worked 
with the Trainer, each receiving feedback in a different manner. 
The following methods were used: 

1. The coaching mode (as soon as the question is presented, a 
light appears next to the correct answer). 

2. The single -error -permitted mode (the student is allowed 
one choice only and then the correct answer is shown). 

3. The practice mode (the student is allowed to continue mak- 
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ing responses until he selects the right one, and then the question 
is changed. 

4. The single-try mode (the student makes one choice, is im- 
mediately informed of the correctness of his answer, without 
necessarily receiving the correct answer, and then the device 
moves on to another question). 

5. The paced practice mode (the student is allowed only a cer- 
tain amount of time to make a response before the new material 
is presented), 

6. The test mode (the student is allowed only one try before 
new material is presented, and there is no knowledge of results). 

It is unfortunate that, although most of the devices employ one 
or the other of the modes, only two limited studies have been 
made as to their relative efficacy. The conclusions reached in 
these studies were that the single -try and the coaching mode pro- 
vided the best learning retention. However, the studies differed 
as to which of these two was the best one to employ. The mode 
used should be the one that will most effectively implement the 
program. For example, the paced-practice mode is used in the 
training devices designed by Gordon Pask to train operators in 
the use of IBM key-punch and adding machines and to teach typing. 

There are two basic methods of giving the student information 
about the correctness of his answers. Some of the devices de- 
signed by the armed services and others depend on lights to show 
the student whether his response is correct or not. This appears 
to be wasted movement. The student must look away from the dis- 
play to find out data on his response, and he must go through the, 
admittedly simple but uneconomical, mental operation of trans- 
lating the lights into the concepts of right and wrong. The major- 
ity of the devices provide feedback data by showing the right 
answer in the display slot. This would seem to be the best method 
because such data should in no way detract from the student's at- 
tention to the program and the stimuli provided. 

In both methods of programing, the feedback data is used either 
to reinforce a correct response or to correct an incorrect one. 
This is done in the branching technique by providing a new expla- 
nation for a misunderstood principle and then sending the student 
back to try to get the right answer (the practice mode). Using the 
Skinnerian technique, the student is allowed one attempt and then 
immediately provided with the right answer (the single -try mode). 
However, some Skinnerian devices do not allow the student to 
continue the program until he has made the correct response. 
This also is a utilization of the practice mode. 

The significance of employing a teaching device in various 
modes can be found by a study of the reinforcement theory of 
learning. To learn, the right answer, and only the right answer, 
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must be reinforced. If the correct answer is made, the student 
receives reinforcement by being allowed to continue the program, 
or, in the Crowder branching method of programing, by receiving 
a programed, verbal "pat on the back/' If the student makes an 
incorrect response, he receives the aversive reinforcement of 
not being able to continue with the program, having the point re- 
explained to him, or such reinforcement as is provided by a de- 
vice such as a light or a buzzer. The more rapid the feedback, 
the more effectively the learning is reinforced and retained. The 
drawback of most manually operated machines is that, in normal 
operation, they exceed the 1/2-1/10 sec optimum interval for 
feedback. 

Reinforcement generally is provided by the success of answer- 
ing a question correctly and being allowed to continue with the 
program. Some of the devices to teach preschool children allow 
for additional reinforcement by dropping out a marble or piece 
of candy for each correct response or a given number of correct 
responses. Most people in the field believe that the amount of 
aversive response should be minimized. Devices that incorporate 
buzzers or lights to reinforce aversively a wrong probably do 
some damage. Additional reinforcement can be provided at small 
extra cost in many devices by allowing for the disappearance of 
questions that have been answered correctly once or twice. 

WHAT OF THE FUTURE? 

There is, at this writing, an established need for improvement 
of the quality and quantity of domestic education. According to a 
report of national goals of the Office of Education, the present 
teacher-to-student ratio in the continental United States is about 
1 : 24 and, over the next 10 yr, the situation will gradually worsen. 
Some of these teachers must be eliminated from consideration 
because of poor qualifications. National Education Association 
statistics show a shortage of 135,000 teachers, 100,000 teachers 
with substandard credentials, and 44 million students enrolled in 
1960, These statistics also indicate that the situation will grow 
worse as time progresses. Existence of a growing scientific body 
of information concerning the learning process and the technolog- 
ical advances in this country should be combined and utilized in 
the education field. Education is still, in the main, oriented to- 
wards the horse and buggy era and has not really begun to make 
the most of our national resources. 

Norman Crowder stated in an interview in the Saturday Even- 
ing Post that "By the mid-1960's most systematic teaching in 
U.S. schools and colleges will be done by teaching machines and 
programed textbooks. " Plans are being presented already for 
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student consoles that will incorporate the following devices: vis- 
ual display screens that use material from a motion or slide pro- 
jector and that will be part of the student console or that can be 
hooked into closed-circuit educational television networks, tape 
recorders for oral language learning and speech therapy work, 
and general -purpose teaching machines that are adaptable to the 
teaching of any number of disciplines. Some special consoles 
might even include oscilloscopes that would enable a language 
student or a speech-therapy patient to study his responses and 
attempt to match them with patterns formed by the instructor's 
responses shown to him on the display screen. These and many 
other devices can be used to aid in the learning process. 



Pseudo-Programed-Instruction Systems 



By LEWIS D. EIGEN 
Center for Programed Instruction, Incorporated 



The application of the principles of programed instruction are 
numerous; they reach to many types of media and a wide variety 
of learning tasks. They can be applied to individuals or large 
groups. In many cases, equipment and procedures are general, 
but specific applications often require specific engineering in the 
mechanical and psychological senses. This article describes some 
of these; there are undoubtedly many others, and the future will 
surely produce a myriad of applications of programed-instruction 
techniques to public education and military and industrial training. 

KEYBOARD TRAINER 

A pseudo -programed-instruction system has been developed 
to teach the manual skills involved in keyboard operation. The 
device, known as SAKI (Solartron-Rheem Automatic Keyboard 
Instructor) was developed jointly by Solartron Electronic Group, 
Ltd. and Rheem Manufacturing Co. for training keyboard opera- 
tors. Almost any type of keyboard operation can be taught by 
means of SAKI. 

Essentially, SAKI consists of three units: a control unit, a 
display unit, and the keyboard (see Fig. 6-2-1). The keyboard is 
identical to the one that the student will use on the job. The dis- 
play unit is composed of two panels that can be illuminated from 
the rear. The lower panel is a diagram of the keyboard. The 
upper panel is a translucent screen on which four rows of char- 
acters can be printed. These characters (numerals, letters, or 
symbols) include the complete range of characters available on 
the keyboard. 

The operation of the SAKI has been described as follows: 

"The pupil sits in front of the machine, and, when it is switched 
on, a light . . . appears behind the first character in the first line 
of the upper translucent panel of the display unit. The . . . charac- 
ter . . . indicates that the pupil is to depress the key representing 
this character. To help the pupil identify the correct key, a light 
also appears behind . . . the appropriate key on the lower display 
panel. When the pupil depresses the key indicated, the lights on both 
display panels will be extinguished, and another light will appear 
behind the second character in the row containing the exercise . . . 
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Fig. 6-2-1. Solartron-Rheem Automatic Keyboard Instructor (SAKE) 

". . . As the response of the pupil becomes faster, the rate at 
which the characters in the exercise are presented by the ma- 
chine is increased automatically. As the pupil becomes even 
more proficient, the lights that appear on the lower display panel 
begin to get dimmer until, finally, they go out ... Sooner or later, 
she will make a mistake and depress the wrong key. Immediately 
. . . the machine slows down and the lights showing the correct 
keys to be pressed reappear. "* 



*A Machine That Learns From Experience, Data Processing, April- 
June, 1959, pp. 108 and 109. 
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Thus, the student receives immediate confirmation of the ap- 
propriateness of his or her response. The machine determines 
with what speed and with what accuracy the student must com- 
plete the first line before being shown the second line of the up- 
per display panel. Many sets of upper display panels are pre- 
pared. When the student completes one, he goes on to the next. 
In general, these sets become progressively more difficult. 

As with the teaching machine, the student is required to make 
an overt response. Unlike programed instruction, however, 
SAKI is not completely student paced. With true programed in- 
struction, the student has control over the speed of his training 
and instruction; neither machine nor program have control over 
the speed with which the material will be presented to him or 
the speed with which he will respond. Even in cases in which 
time allowed for training is a critical factor, the pacing is rela- 
tively immaterial. In training keyboard operators, however, 
pacing is critical in that the desired terminal behavior, which is 
the ability to operate the keyboard, is judged in terms of both ac- 
curacy and speed. 

In most verbal learning, and in many manual skills, it is im- 
material how fast the actual response is made. Within limits, 
the duration of time that elapses between the stimulus and the 
response is also unimportant. For example, if a salesman is 
asked by one of his potential customers about the characteris- 
tics of one of his products, a second or two delay in giving his 
response is not at all detrimental to his sale. In the case of the 
keyboard operator, however, a one-second delay in making each 
response effects the efficiency of the entire operation. Hence, 
in the case of keyboard operation, speed is an integral part of 
the training task. 

Pacing in the SAKI is an interesting combination of student and 
machine pacing. Preset standards of accuracy and speed must be 
reached for mastery, but the manner in which the student reaches 
these standards depends on the speed and accuracy which he de- 
velops during the learning task. 

PRODUCTION-LINE TRAINER 

Another type of system that has many of the characteristics 
of programed instruction has been developed for use in industrial 
production. The objective of this system is to produce work of 
high quality under controlled conditions. This type of system 
consists of a small audio-visual unit that presents pictures in 
the form of projected 35mm slides with a concomitant audio 
message presented through earphones. The machine informs 
the employee visually and orally of the production procedure 
he is to carry out. The employee then performs the work in the 
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prescribed manner. When he decides that the work is completed, 
he goes to the machine to be informed of the next step to betaken. 
One of the features of a device like this is that it can be paced by 
the student or by the machine. In other words, the employee can 
control or be controlled by the delay between steps in the produc- 
tion sequence. 




Fig. 6-2-2. Hughes Videosonic Teaching System 

Devices of this type have been manufactured by Hughes Air- 
craft (see Fig. 6-2-2), Applied Communications Systems divi- 
sion of Litton Industries, and Graf lex and LaBelle, just to name 
a few. Like teaching machines, these devices also must be pro- 
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gramed. In particular, the production task must be broken down 
into small increments and presented to the employee so that he 
can perform the appropriate task with a minimum of error. 

The usual type of reinforcement, that is, confirmation of cor- 
rect response, generally is absent in this type of system. The 
success of the program is almost completely determined by the 
skill with which the programer has broken down the steps and 
successfully used strong prompting techniques. It should be 
emphasized that the prompting techniques, as contrasted with 
the usual teaching-machine techniques, need not be withdrawn. 
It is not necessarily desirable to have the employee perform 
without the aid of the machine, though in point of fact, this would 
probably happen after a while. The whole program can be highly 
prompted, and recall is not important because the machine is 
always present. This latter fact makes the job of programing 
considerably easier in many ways; the withdrawal of cues and 
provision for reinforcement contingencies are not necessary. 

This type of system is of special value for production of 
equipment that is fast becoming obsolete. Procedures along 
the entire assembly line can be changed from day to day for 
the new improved version of the product merely by programing 
the changed aspect and replacing the old slides and tapes. 

It also should be pointed out that this type of device also can 
be used as a Skinner-type audio-visual machine. Thus, it can 
be used in training tasks for which response to an audio stimulus 
is the desired terminal behavior. Training of telephone operators 
would be an example of this. The tape can simulate a customer 
calling the operator, and the visual panel can presentform charts 
and reference material that the operator would use in dealing 
with the customer. The operator can make some sort of response 
and then receive confirmation as to the appropriateness of the 
response on the next slide and next audio message. 

TROUBLE -SHOOTING TRAINER 

Another type of system that has been used for instructional 
purposes is the ' 'trainer-tester". The " trainer-tester" consists 
of a printed sheet that might have a paragraph stating the trouble 
for which repair would be necessary, a list of possible remedies 
(for example, "Replace tube V-5.") and, next to each possible 
remedy, an aluminumized tissue that has been pressed on top of 
the paper. The trainee can erase the tissue opposite any of the 
possible remedies. When he does, information is revealed as to 
whether or not the instrument would operate properly with the 
particular remedy chosen. 

The student decides what remedies to select by studying a list 
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of symptoms. The list of symptoms consists of numbered connec- 
tions in the circuit where meter readings might be taken. Next 
to each possible meter-reading point is an aluminized strip that, 
when erased, reveals the meter reading that would be taken at 
that point in the defective instrument. For example, the trainee 
might decide to measure the potential drop across resistor V-5. 
The student then erases the appropriate portion of the aluminum 
strip and discovers that the potential drop across that resistor 
is 3.2. On the basis of this data, he makes a decision as to what 
type of remedy might be appropriate and then makes an erasure 
in the list of possible remedies. When the trouble has been 
"fixed," he is so informed. Thus, the trainer-tester technique 
provides practice in trouble shooting without requiring expensive 
and possibly unreliable equipment. 

The reinforcement contingencies that can exist with the 
" trainer -tester " depend on the way in which it is programed. 
On initial practice sequences, if the student erases an aluminum 
strip in the list of symptoms that would give him data irrelevant 
to the trouble, the program might inform the student that he 
chose irrelevant information. Later, this could be "faded out" 
so that he would get some data, regardless of where he had 
erased in that section. He would, however, receive reinforce- 
ment as to the appropriateness of his choice until he had made 
an erasure in the list of remedies. 

Different military organizations have used the "trainer-tester" 
in experimental situations . The following quotations concern the 
"trainer-tester" report of the officer in charge of U.S. Navy 
Guided Missiles School at Virginia Beach: "A definite saving in 
time is accomplished in that each trainee may be assigned the 
same trouble and work at his own speed . . . Where there is no 
need to physically install the trouble in the equipment, there is 
always a savings of time. A saving in material results in not 
having to solder and unsolder connections . , . Wear and tear on 
parts and equipment for which the "trainer-tester" is substituted 
is reduced to a minimum since the equipment need not be used 
in trouble shooting. 

"Due to the fact that so many different troubles can be intro- 
duced to the trainee in a given length of time, he can develop 
definite habits for trouble shooting that improve his ability to 
analyze troubles and proceed in logical steps to their solution 
and correction." 

Preliminary data from the atomic -weapons -training group at 
Sandia Base in Albuquerque, N.M. indicate that the mean time 
per problem for correct action on the "trainer -tester" simulator 
was 13 min. The training task thus is facilitated by enabling the 
trainee to avoid the soldering and unsoldering of connections 
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and all those time-consuming activities that are not directly 
related to the problem at hand. 

The use of the "trainer-tester" is certainly not limited to 
electronics trouble shooting; it can be used for trouble shooting 
in other areas such as engine maintenance and repair and prob- 
lems of logistics management. It also could be used as an an- 
swer sheet for a standard multiple -choice -type teaching-machine 
program eras ing the appropriate aluminum strip would confirm 
the fact that the student had selected the appropriate response. 

COMMON CHARACTERISTICS 

The aforementioned three types of systems have several char- 
acteristics in common with programed instruction. It should be 
noted that each of these techniques developed from a need to 
solve specific categories of training problems. In the case of 
SAKI, the training problem was keyboard operation. In the case 
of the "trainer-tester," the general area was trouble shooting. 
Whereas programed instruction can be applied to many areas of 
training, the three aforementioned pseudo-programed systems 
are limited in their application. SAKI, for instance, could not be 
used in the training of salesmen. The trainer-tester technique 
would not be applicable for the training of keyboard operators. 

One reason for these specialized techniques being developed 
is that programed instruction is not a suitable solution. Most of 
the techniques in programed instruction have been developed to 
solve problems in verbal training and education. In general, re- 
inforcement for appropriate responses is provided for in a ver- 
bal type of program by having students make the response and 
compare it (or have it compared by a machine) with the pro- 
gramer's version of the correct response. It is extremely in- 
efficient, for instance, for the keyboard-operator trainee to do 
this. In point of fact, this is exactly how a person learns to op- 
erate a keyboard without any training program whatsoever; he 
strikes a few keys, looks at the sheet of paper in the typewriter 
or the IBM card and determines if, indeed, this is what 
he wanted to type. 

PROVIDING REINFORCEMENT 

The two major problems in utilizing programed instruction 
for any type of manual training, whether it be keyboard or pro- 
duction, are in providing for proper reinforcement contingencies 
and in providing for branching possibilities with a number of 
alternative actions far greater than the usual three, four, or five. 
In training for tasks that involve the use of specific equipment, 
it is usually very difficult to provide for the proper reinforce- 
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ment contingencies. It is easy to tell the trainee, for example, 
to "clamp the piece of work into the lathe and turn the clamp 
tightener marked A in your diagram/' but we are not at all sure 
that what he tightens is the clamp tightener. There usually is no 
method for the trainee to get feedback from the device as to the 
appropriateness of any phase of the operation. This is particu- 
larly true when operation of the device requires many subopera- 
tions before the trainee can tell whether or not he has done the 
right thing. 

So far, there have been two methods developed to handle this 
type of problem. The first is designing a piece of equipment that 
will sense the appropriateness of the response; this piece of 
equipment then could be used in conjunction with the unit that the 
trainee is learning to operate. (SAKI is an illustration of the use 
of this technique.) But the use of this technique often requires 
intricate sensing instruments that must be specially devised.^ 
This is usually expensive and, in many cases, not justifiable in 
terms of the training value gained for the dollar expended. 

The "trainer-tester" is a simple example of the second method 
-use of simulators-to solve this problem. Simulators and so- 
phisticated sensing devices have been combined with very ef- 
fective results (the Link trainer is an excellent example of this) . 
The combination is usually quite expensive, and it is certainly 
more desirable to use a technique such as the "trainer-tester" 
if it is applicable. For training in the use of many pieces of 
equipment, the trainer-tester technique is more efficient, but it 
usually would not be used to develop many skills. 

A third possible solution to the problem is to use a modified 
version of the device that the trainee is being taught to operate 
in conjunction with a standard teaching-machine-type program. 
The device would be essentially that which the trainee would be 
taught to operate but would have modifications so that the rein- 
forcement he receives can be provided for in the program. An 
example of this technique is a program to teach elementary- 
school students how to use a microscope. Such a program has 
been written at the Center for Programed Instruction and has 
proved extremely effective in developing the necessary skills 
on the part of the students for microscope operation. 

The program is used in conjunction with a specific training 
microscope. This instrument essentially is like the microscope 
the student would normally use except that the different parts 
have been color-cued, that is, each essential manipulative part 
of the microscope has been painted a different color. Thus, the 
program can now say to the student: "Put your hand on the fo- 
cus knob. What color is it?" The student can respond by writ- 
ing the word for a color in an appropriate answer space and 
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then receive confirmation as to the appropriateness of the word 
he wrote. If the focus knob were blue and his response were the 
word "red", then the correct answer that would be given in the 
program would be: "Blue, if you wrote any other color, you have 
the wrong part." 

The instrument or parts thereof can be modified so that the 
student can do his own sensing. For example, if the student were 
instructed to focus a normal microscope and slide, we would have 
no way of knowing whether or not he did focus the instrument, but 
if we use a specially prepared slide with the microscopic words 
"in focus" printed on the slide, we can ask the student to turn 
the focus knob until he can read the word. We can then make the 
assumption that when he can read the words "in focus," he has 
successfully focused the microscope. 

PROVIDING ALTERNATE RESPONSES 

The second major problem is that of providing a large number 
of possible responses. Theoretically, a Crowder-type scrambled 
program could provide for all possible responses, but this would 
necessitate an extremely bulky program, and its sheer size would 
be overwhelming. Its large storage capacity and easy access to 
the storage makes the electronic computer a natural solution to 
this problem, but this solution involves considerable expense. 
Thus, the trainer-tester technique seems to give the most effi- 
cient solution to this problem when one considers the time, ef- 
fort, and expense. Unfortunately, however, the "trainer-tester" 
cannot be applied to many important training situations. 

GROUP INSTRUCTION 

The discussion so far has been concerned with individual 
instruction. There are many possibilities of using some of the 
principles of programed instruction for group instruction or 
mass communication. One such system has been developed by 
Teleprompter Corp. for lecture halls and closed-circuit television. 

Programed instruction usually features the overt response 
of the learner, and the programer can adjust his behavior (that 
of writing the program) on the basis of the student's responses. 
Teleprompter has developed an "answer back" pushbutton con- 
trol unit (see Fig. 6-2-3) by means of which each viewer through- 
out the entire closed -circuit system can respond to multiple- 
choice questions shown on the screen. The same type of "answer 
back" control units can be installed in seats in the lecture hall 
so that each member of the audience has the opportunity to make 
an individual response to the questions presented. Thus, the pro- 
gramer or lecturer can change his behavior (adapt the program) 
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Fig. 6-2-3. Teleprompter "answer back" pushbutton control unit 

on the basis of the student responses, which he can read off a 
percentage meter either at the lectern or in a TV studio. It is 
also possible, for mass-viewing purposes, to plug each of the 
"answer back" control units directly into a computer that can 
score tests and/or do some sort of item analysis for program- 
evaluation purposes. 

Another characteristic of the Teleprompter system in lecture 
halls is the preplanned program. The lecture hall can be equipped 
with several random-access, rear -screen projection units, which 
can project combinations of images on the screen, and a rear- 
screen motion picture projection unit or, if desirable, a closed- 
circuit, rear-screen-projected television set. The lecturer can 
control the presentation by means of a specially wired lectern. 

It is also possible under the Teleprompter system to have the 
lecturer read his material from a continuously moving Tele- 
prompter in the lectern. It is further possible to have a presenta- 
tion of the different visual materials completely controlled by 
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the movement of the teleprompter script; when the lecturer 
reaches a certain point in his presentation, the appropriate ma- 
terial will be flashed on the screen. As a further sophistication 
of the system the lecture, as well as its controls for visual ma- 
terial, can be put on tape, and instruction can be given to a large 
group of students without the physical presence of the instructor. 
The Teleprompter system has been used by several of the major 
universities in the United States, and the University of Wisconsin 
in particular has experimented with presenting fully controlled 
tape recorded lectures through the system (see Fig. 6-2-4). 




Fig. 6-2-4. Teleprompter installation at the University of Wisconsin 

The United States Army also has used Teleprompter installa- 
tions for educational purposes at Redstone Arsenal. Experi- 
mental data from the Ordinance Guided Missile School at Red- 
stone have shown that, in an acquisition and computer course 
for the Nike-Hercules missile, there was no significant differ- 
ence between the so-called " conventional' 7 courses and those 
taught with the Teleprompter system, but there was a time sav- 
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ings, in using the Teleprompter system, that ranged from 16% 
with some units to 41 % with others. * 

Although the Teleprompter system has many of the advantages 
of programed instruction, it also shares some of the disadvan- 
tages. Relatively high programing and production costs for the 
system can be justified only in terms of the number of students 
reached and/or the number of times the presentation is given. 
Unlike programed instruction, however, the Teleprompter sys- 
tem retains one characteristic of any type of group instruction, 
that is, students can not proceed at their individual paces. Never- 
theless, this type of system demonstrates how equipment can be 
designed and utilized so that many of the inherent disadvantages 
of group instruction can be overcome. Although it does not re- 
tain all of the aspects of programed instruction, the Teleprompter, 
as a pseudo-programed-instruction technique, does provide many 
of the advantages of true programed instruction. 



* Training by television and television prompting equipment, Army Ord- 
inance Guided Missile School, Redstone Arsenal, Alabama, February, 1959. 



The Influence of Teaching-Machine Technology 
on Electronic-System Maintenance Training 
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Our society is moving continually in the direction of complex- 
ity. This complexity thrusts upon us the responsibility for under- 
standing and controlling it. Teaching-machine technology sug- 
gests a way to more quickly assist us in acquiring the knowledge 
and skills we must have in an increasingly complex society. 

Of all the machine systems with which we are familiar, one 
kind in particular is troublesome because of its increasing com- 
plexity: electronic systems. Electronic systems seem almost 
to generate intrinsic problems as they satisfy extrinsic specifi- 
cations. This article describes a continuum of developments 
dealing with electronic -system maintenance that have been in- 
fluenced by performance requirements, studies of behavioral 
patterns, technical-training requirements, and engineering- 
design and management philosophies. In the discussion, refer- 
ence will be limited to the intrinsic problems of electronic- 
system maintenance that are created as new and different equip- 
ments and instruments are produced. 

A MAN -MACHINE RELATIONSHIP 

FOR MAINTENANCE 

There are at least two ways in which one can think about main- 
tenance: designing maintainability into the system at its concep- 
tion and shaping the behavior of human maintainers to provide 
maintainability after installation. This delineation follows closely 
the basic ideology of Taylor with respect to man-machine sys- 
tems involving the operator of the machine rather than the main- 
tainer (Taylor, 1960). He reminded us that it was during World 
War II that the approach of designing the task to fit the operator 
was added to the more traditional psychological procedures of 
selecting and training operators to fit their jobs. This was ne- 
cessitated by the variety and complexity of military equipment. 
Machinery had finally outran the man's ability to adapt, and the 
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recognition of this fact was, in Taylor's view, the first impor- 
tant insight in the development of engineering psychology. It is 
regrettable that very little emphasis was given to designing the 
equipment for maintainability. 

The man-machine system concept probably grew from an ex- 
tension of the weapon systems and control systems notion in engi- 
neering. And as Taylor has said, the concept made obvious the 
fact that one could not design the mechanical portions of a sys- 
tem of which the man was a part without knowledge of the char- 
acteristics of the human component for operating purposes. His 
analysis of man-machine systems into two areas, each dealing 
with a different objective, is worthy of review; one class of sys- 
tems is constructed to affect the condition of the human in that 
system. An example is the learner and his teaching machine. 
The purpose of inserting the nonhuman element into the system 
is to improve the well being or efficiency of the human within the 
system. The other class of systems is conceived as one which 
alters the environment or changes relationships between the sys- 
tem and other entities external to it. The pilot and his plane illus- 
trate this class. With these systems, Taylor believed the depen- 
dent variable of greatest interest is a measure of the performance 
of the total system and this measure reflects the combined per- 
formance of all the components in the system, not that of the hu- 
man alone (Taylor, 1960). 

It is obvious that the human component of the man-machine 
systems defined by Taylor involves human operators and not hu- 
man maintainers. While he did feel that in addition to controlling, 
the human collects information, filters, stores, and evaluates it, 
applies rules to it and makes decisions, these functions were in 
the operational frame of reference. One might suggest that a 
man-machine relationship might exist in the interaction between 
a system and its maintainer particularly with reference to fully 
automatic systems which have no human operator but which do 
have periodic preventive maintenance or malfunction maintenance 
performed. 

THE DESIGN ASPECT OF MAINTENANCE 

There has been some awareness by engineering administrators 
and designers of the total problem of electronic systems mainte- 
nance in terms of maintainability and reliability (see Folley and 
Altman, 1956). This approach, of course, deals with equipment 
configuration rather than personnel training. Folley and Altman 
identified solutions to this problem by recommending design prac- 
tices which would maximize the ease with which electronic sys- 
tems can be maintained. The self -test circuit and checkout con- 
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sole and the computer diagnostic routine, fault isolation program, 
and test routine are recent examples of design techniques to im- 
prove the engineering aspect. It was felt in 1956, at least, that 
maintainability requirements for electronic equipment were ex- 
tremely vague as compared to the detailed performance require- 
ments usually specified (Folley and Altman, 1956). In referring 
to trouble shooting procedures, Folley and Altman believed that 
systematic procedures should be provided since technicians 
would frequently use inefficient or even dangerous practices if 
efficient procedures were not provided. 

The problem described by Folley, Altman and others concerned 
with maintainability in the armed forces technical organizations 
was anticipated somewhat earlier by several practitioners who 
also had visibility. Notable among these was Miller, who, in 1953, 
summarized several prior investigations concerned with both the 
design and personnel aspects of system maintainability (Miller, 
1953). With regard to design, he contended that the following 
three principles governed equipment design: 

1. The design of new equipment is also the design of new jobs 
(operator and maintainer). 

2. Equipment performing complex operations can be designed 
for easy operation and easy maintenance). 

3. Dependability and maintainability in realistic field condi- 
tions should enter into the decision to approve designs. 

He was of the belief that planned simplicity in the maintainer 's 
job needed to be stepped up and made more systematic. 

Nadel, in his capacity as Secretary of the Advisory Panel on 
Personnel and Training Research for the U. S. Department of 
Defense, wrote in 1955 that the maintenance of electronic equip- 
ment was a major military operation for each Service, and that 
equipment failure could mean the difference between victory and 
destruction. Equipment failures, he pointed out, could be attrib- - 
uted as much to the very complexity of the equipment itself as 
to other factors (Nadel, 1955). 

In general, some effort has been made to improve maintain- 
ability through design. Equipments with digital computer sys- 
tems or subsystems are often easier to trouble shoot in so far 
as localizing the malfunction is concerned, if the proper mainte- 
nance routines are programed. But the effectiveness in trouble 
shooting still appears to be largely a matter of individual human 
skills, according to Ely, Hall, and Van Albert (1960). Trouble 
shooting remains in the province of human decision and action, 
in the opinion of the author, and will continue there for awhile, 
at least so long as the equipment which trouble shoots a system 
will itself need to be maintained. 

And what portends for the future when man is in space? Jones, 
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in his remarks on Mercury at the 1959 American Psychological 
Association convention (APA, 1959), pointed out that studies were 
being conducted to reveal the types of difficulties that are apt to 
occur before and during a mission by estimating what failures 
are apt to occur in the systems and components. The mission is 
automatic, and the only time the human has a significant role 
beyond the scientific one is when there are problems in system 
operation. Failures of the component, effect on system operation, 
sensory cues, instrument cues, and corrective actions may be 
available both to the ground personnel and the astronaut. These 
can be used, Jones avers, to determine the procedures that the 
human must accomplish and, probably most important, to de- 
fine the nature of the diagnostic tasks he must perform. Since 
similar constellations of signs may appear for fifty or sixty dif- 
ferent failures, these must be differentiated to decide the proper 
corrective action for the specific failure. 

THE PERSONNEL ASPECT OF MAINTENANCE 

Assuming that the engineering designers will make a super- 
human effort to produce reasonably maintainable equipments, 
we can now turn from the non-human to the human category and 
be concerned with the maintainer, leaving a discussion of the op- 
erator for another time. Miller's initial interest in the early 
1950' s centered about the development of methods for predicting 
and anticipating job requirements for maintainers. Accurate fore- 
casts would suggest early training programs so competent main- 
tainers would accompany the system to its operational environ- 
ment. The Air Force AN/APQ-24 Radar and the K-l Bombing- 
Navigation System were used for early studies (Miller, 1953). 
Some of the significant observations he made in 1953 were as 
follows: 

1. There is a need for early knowledge of the job activity or 
behavior for maintenance through systematic analysis. 

2. Knowledge of job activity produces information dealing with 
job relevance. 

3. Training devices and instructional materials can be pro- 
duced from job activity and job relevance data. 

4. Ability to read technical orders and translate them into job 
action is a requirement common to all technical jobs. 

5. Knowledge basic to design is not necessarily basic to main- 
tenance. 

6. Research is necessary to determine the information most 
effective in performing maintenance and the ease in learning this 
by the rnaintainer. 

7. A complete, clearly written set of job instructions may actu- 
ally take the place of training. 
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8. A major revision of current technical vocabulary with sim- 
plicity as an objective is necessary. 

9. Specialists in human factors and in training should assist in 
planning the equipment design for maintenance. 

This series of studies which began in 1950 and was terminated 
in 1956 contributed to the newly emerging systems maintenance 
philosophy by identifying problems and proposing solutions. The 
Air Force Qualitative Personnel Requirements Information (QPRI) 
program, in its formative stages, benefited from this effort. It 
was concluded that job requirements for maintainability could be 
forecast and used as a basis for planning personnel and training 
programs in an orderly manner. However, the studies suggested 
that additional investigation was required in developing procedures 
whereby contractors could provide for earlier planning through 
more intrasystem coordination in the development of design speci- 
fications, functional data-flow diagrams, and ground support 
equipment (Miller, 1956). 

THE 1955 SYMPOSIUM ON ELECTRONICS MAINTENANCE 

In August 1955, the Department of Defense conducted a sympo- 
sium bringing together practitioners in the military systems 
maintenance field. Hall, of the Navy Electronics Laboratory, ex- 
pressed the view that the operational effectiveness of man-machine 
systems may be considered as a multiplicative function of the 
ideal potentialities of the equipment (potentialities as viewed by 
creating engineers), performance potentialities of the human op- 
erator using the equipment, and effective maintenance. He cited 
a 1950 study which showed that only 33% of Navy electronic equip- 
ment was operating satisfactorily at that time, and disclosed that 
9% of electronic equipment in Fleet Units was inoperative and 
65% was operating poorly or improperly in 1955. Hall thought 
that the problem of millions of failures was serious not only be- 
cause of the cost in billions of dollars but also because of the de- 
crease in operational effectiveness and provided these facts to 
support his view: a certain type of complex airborne electronic 
equipment required over three hours of maintenance for each 
hour of flight; one-third of the Air Force operating cost is for 
maintenance and one -third of their personnel is engaged in this 
activity, although over half of their maintenance is done by con- 
tractors; the cost of maintaining electronic equipment in the Air 
Force is about twice as much per year as the initial cost; main- 
tenance cost during the life of Navy equipment is ten times its 
initial cost (Hall, 1955). 

Carstater, of the Bureau of Naval Personnel, pointed out, in es- 
tablishing a frame of reference, that while proficiency in trouble 
shooting was probably the hallmark of the electronics maintenance 
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technician, the symposium was concerned with electronics main- 
tenance activities such as inspection, calibration, adjustment, as 
well as trouble shooting. In a very loose sense, he thought, 
trouble shooting and repair are necessary only when other main- 
tenance activities have failed (Carstater, 1955). 

In the opinion of Irvine and Smith of the Navy's Personnel Re- 
search Field Activity at San Diego, the personnel problems in- 
volved in maintaining Navy electronic devices differed from those 
involved in maintaining devices in the Army or Air Force. Land- 
based organizations were able to have their technical personnel 
specialize in the kind of work and in the type of equipments, but 
the sea-going Navy was limited in the amount of shredding out 
(organizational specializing) that could be done. Space limitations 
and relative isolation of the ship as an operating unit require a 
small number of highly skilled electronics technicians to main- 
tain a wide variety of electronic equipments. The problem in 1955 
of properly maintaining the Navy's electronic systems was due 
primarily to a shortage of trained technical personnel the result 
of a shortage of qualified civilian recruits, the long training pe- 
riod necessary to qualify, and the low re -enlistment rate. The 
problem was summarized by saying that "too much is required 
of too few" (Irvine, 1955). 

Earlier, it was pointed out that the functions of operator and 
maintainer were uniquely distinct. However, the Navy did identify 
certain maintenance functions with the human operator (Irvine, 
1955). This practice reemphasized the need for a more analytical 
approach to the job to be performed rather than the man who per- 
forms the job. There had been too little concern with the need to 
study the job independently of the man. There was an awareness 
of the need to determine the knowledges and skills required for 
each essential task, then sort the tasks requiring similar knowl- 
edges and skills into functional groups or packages. This con- 
firmed Miller's observation in 1953 that detailed knowledge of job 
activity was essential (Miller, 1953). It will be seen that the con- 
cept of job analysis, which is essentially the result of studies of 
job behavior patterns, seems to permeate the discussions and 
studies of maintainability down the years to and including 1961. 

The Army in 1955 did not face this problem with the same mag- 
nitude as did the other services, yet the awareness of solutions 
did exist. Karcher, of the AGO's Personnel Research Branch, 
believed that while selection procedures and similar personnel 
research efforts of long-range orientation should not be over- 
looked, research in job analysis, training content, and training 
methods would have a more immediate payoff (Karcher, 1955). 

Gagne, of the Air Force Personnel and Training Research Cen- 
ter, in reviewing the work of Miller and others in AFPTRC, said 
that the Air Force development philosophy was that a weapon 
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system is a man -machine system and that many detailed direc- 
tives had been issued which have the aim of ensuring that plan- 
ning for the man would go hand-in-hand with development of the 
equipment. The importance of determining the so-called Qualita- 
tive Personnel Requirements for newly developed weapon sys- 
tems was apparently receiving extensive recognition throughout 
the Air Force. He summarized his thinking about job analysis in 
the following terms (Gagn, 1955): 

1. Information about job requirements for new equipment 
comes from equipment designers who have in mind but not nec- 
essarily in recorded form the activities required for mainte- 
nance. Recording this systematically during early development 
remains a problem. 

2. Job requirements for maintenance can be predicted early 
in the development of new equipment, and these tend to remain 
stable as production is reached. The points of correspondence 
between predicted job behavior and actual job behavior remain 
to be explored. 

3. Terms used in describing jobs must be understandable, reli- 
able from job to job, and meaningful for selection and training. 
To meet these criteria requires a detailed, behavioral descrip- 
tion. What seems to be required and yet undeveloped is a set of 
operational definitions of behavioral categories. 

The Navy had prognosticated the problem of job behavior pat- 
terns very early in the spectrum of time and engaged members 
of the Department of Psychology of the University of Southern 
California to intensively study this area. Actually, the Navy De- 
partment had used crude job analysis techniques in the early days 
of World War II to establish training programs for existing and 
newly emerging trades, crafts, and technical occupations. The 
author, when faced with the task of establishing a technical train- 
ing program for Navy civilian employees in June 1942, adapted 
trade and job analysis techniques used in vocational schools for 
this purpose (Silvern, 1948 and Greenly, 1943). There is evidence 
that Allen, between 1910 and 1919, and Selvidge in 1923, published 
occupational analyses in the form of jobs and operations compos- 
ing jobs from their experiences in World War I (Graham, et al., 
1943 and War Manpower Commission, 1945). This was succinctly 
stated in 1943: "instructional material for all vocational teach- 
ing should be based upon an analysis of acceptable performance 

of what a practitioner of the vocation does the testing program 

involves three types of tests oral test is a quick method of 

checking the why of the job written test to measure the re- 
lated technical information performance test measures skill, 

the ability to do, along with how to do it " (Graham, et al., 

1943) and again in 1945: "job analysis or the making of job break- 
downs is an important preliminary step before instruction" (War 
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Manpower Commission, 1945). One can readily conclude that job 
analysis was not really new, but the techniques of obtaining de- 
pendable job behavior information and detailed job requirements 
in the electronic systems era in sufficient time to support the 
systems operationally had yet to be formulated. 

Bryan, of the University of Southern California, presented at 
the 1955 Symposium an evaluation of various job analysis methods 
as an outgrowth of Navy-sponsored research there (Bryan, 1955b). 
It was his contention that surprisingly few comparisons of job 
analysis methods had been published and that published material 
contained descriptions of general procedures. When an investi- 
gator needs to define, describe, or assess the requirements of a 
particular job, it may be necessary to develop new job analysis 
methods or to modify existing ones. 

As a result of research in using a number of job analysis tech- 
niques, he had concluded the following: 

1. No single method will produce all the information required 
to adequately describe a job. The compromise is a combination 
of methods which supplement information that can be collected 
easily with a large measure of concentrated on-the-spot observa- 
tion. 

2. There is definite need for more intensive study of specific 
methodological problems inherent in job analysis in order to 
specify more completely the limits of each method. 

3. The role of job analysis in our modern complex organiza- 
tions is too crucial to allow its methods to remain in the realm 
of the arts . 

The 1955 Symposium, while very clearly concerned with job 
analysis, was in agreement that maintainer training techniques 
could spell the difference between success and failure. Wischner, 
Barch, and Hammock of HumRRO concluded that Army techni- 
cians regard school training highly; after experience on the job, 
can offer constructive suggestions concerning the relevance of 
the school curriculum; needed more training in systematic 
trouble shooting; needed more training in practical trouble shoot- 
ing exercises, routine checks, adjustments, and location of parts; 
and more practice in use of block diagrams (Wischner, et al., 
1955). Technicians found circuit analysis and equipment construc- 
tion instruction of low job applicability. One of the most signifi- 
cant general findings was the virtual absence of systematic on- 
the-job training programs in any of the maintenance areas studied. 
In addition, the Navy experience with operators performing cer- 
tain maintenance functions was supported by the HumRRO team 
which noted that of all the malfunctions obtained, in approximately 
50% symptoms were first observed and reported by the operators 
(Irvine and Smith, 1955). These operators also assist the me- 
chanic in preventive maintenance. 
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Carstater, of the Bureau of Naval Personnel, concluded that 
there is a very real need to obtain greater continuity between 
school training and on-the-job learning (Carstater, 1955). Much 
that is known about how men learn, how to teach, course con- 
struction, school plant design and operation, supervision of in- 
struction, adaptation to individual differences, diagnosis and re- 
medial instruction much that is available in the science and art 
of teaching is not being effectively brought to bear in solving the 
training problems of the Military Services. 

Carp, of the Air Force Personnel and Training Research Cen- 
ter, in a discussion following a paper, said "I am quite ready 
and willing to take an extremely critical look at most of our train- 
ing activities and ask the question, 'Are they really training for 
the job?'" (Carp, 1955). 

THE 1956 FIRE CONTROL SYSTEMS TRAINING CONFERENCE 

On 2 May 1956, a Fire Control Systems Training Conference 
was conducted by the Air Force at Lowry Air Force Base (USAF, 
1956). McKinney, of Air Defense Command, had established a 
correlation between training and the overall results of the fire 
control system and asserted that maintenance was handicapped 
by the extreme shortage of maintenance personnel. He compared 
fire control systems maintenance training with the weather, say- 
ing that everybody talks about it, but nobody does anything about 
it. There has been talk for four years, but very little concrete 
action to improve it. 

Up to now, no mention had been made of the human instructor 
aspect of maintainer training. Turpin, of Lowry Air Force Base, 
directed his attention to this problem echoing similar experience 
of varying magnitude in the other services. He believed the in- 
structor corps was frightfully inexperienced, both technically and 
professionally, and this was attributed to turn-over rates; i.e., 
failure to re -enlist and transfer to other Air Force units. Under 
circumstances in 1956, there was no relief in sight. He suggested 
that maintenance of complex electronic systems was not beyond 
the learning ability of most of the airman recruits, but that de- 
velopment of skills in both mechanics and technicians was fast 
becoming beyond the teaching abilities of instructors. Unless a 
change in the way instructors were selected and developed, class- 
room efforts, in his view, appeared doomed to a plateau of medi- 
ocrity. It seemed to him, as a civilian educator, that too many 
failed to realize that a successful instructor must acquire a dual 
proficiency in the job performance he teaches, and, from an edu- 
cational point of view, in the developmental teaching process. 
The author had recognized the dual role of the instructor as de- 
scribed by Turpin and had identified knowledge of subject and 
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communication as instructional ability in a course in training 
technical instructors for the Navy Department and had refined it 
into technical competence and instructional competence by 1956 
in a textbook dedicated to engineers who were called upon to in- 
struct electronic systems maintenance without prior instructional 
experience or training (Silvern, 1948 and Silvern, 1957). Later, 
in 1957, this had been further analyzed to technical competence 
(knowledge of engineering, weapons systems and lesson) and 
communicative competence (use of lesson plan, integration of 
learner materials, participation, communication, knowledge of 
trainees, integration of training devices). In addition, managerial 
competence and personal characteristics were similarly de- 
scribed (Silvern, 1960). 

Gagne, representing AFPTRC, raised an interesting question 
at the conference. Would it not be preferable, he said, in view of 
the heavy demands on skilled manpower which modern weapon 
systems make, to be sure that all possible means have been ex- 
ploited to make the learning of these skills a fully effective proc- 
ess? Would it not be preferable if the limits of what training 
could accomplish were more certain? He announced that plans 
to conduct research to determine principles governing the effec- 
tive use of training equipment, particularly self -instructional de- 
vices (teaching machines), in the area of MG-10 maintenance 
training had been formulated (Hughes, 1956 and Briggs and Duvall, 
1957). The basic requirements for these might be the following: 

1. Measurement of the student's performance. 

2. Exploitation of various attractive ways of feeding back this 
information to the student. 

3. Maximal use of the student's natural interest in competing 
with himself or with his fellows. 

Gagne had reason to believe that this simple everyday kind of 
motivation could make a. tremendous difference in students' per- 
formance. 

At the conclusion of this conference, Renshaw of HQ, USAF, 
felt that the Air Force was placing increased emphasis on the 
critical problems which apply not only to fire control systems 
maintenance but to all complex equipment coming into the inven- 
tory in the future. 

It would appear that the equipment was becoming more complex, 
at least in terms of maintainability, the instructor corps did not 
generally meet the total requirements of the services for one 
reason or another, and much was yet to be learned about job be- 
havior in the tangible form of job analyses. 
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NAVY RESEARCH AT THE UNIVERSITY OF 
SOUTHERN CALIFORNIA 

The Office of Naval Research by 1950 had reached the conclu- 
sion that the job behavior of Navy electronics technicians, partic- 
ularly when involved in localizing malfunctions or trouble shoot- 
ing equipments and systems, was still not well understood either 
by engineers or psychologists. In view of the alarming increase 
in the electronic systems inventory, the Department of Psychol- 
ogy at the University of Southern California was asked to organize 
a research effort and, in the eight-year period beginning late in 
1952, over 30 technical reports were published dealing with the 
conclusions of very carefully organized and controlled experi- 
ments. This is an ongoing general program to analyze objectively 
the job of the naval electronics technician, but it was quickly 
found that the trouble shooting aspect of job behavior was a 
highly desirable skill which required special training and special 
study. 

Much of the early effort consisted of shipboard observation of 
electronics personnel while performing their duties in the des- 
troyer environment. As a result, several tests were designed by 
1953 which measured various aspects of trouble shooting profi- 
ciency. One of these, the MASTS, was the multiple alternative 
symbolic trouble shooting test. Later, with a mechanism for 
automatically recording the subject's responses, it became the 
AUTOMASTS. In essence, these tests replaced both electronic 
equipment and test instruments in a trouble shooting situation by 
the use of a symbolic format. In March 1955, it was reported by 
the Electronics Personnel Research Group that the experimental 
evidence thus far suggested these conclusions (Bryan, 1955a): 

1. A test battery had been developed which was useful and de- 
pendable in selecting a criterion of electronics technician techni- 
cal competence. 

2. Identification of trouble shooting goodness is related to 
availability of adequate performance records such as produced 
by AUTOMASTS. 

3. Electronics trouble shooting is a very complex process. 

4. Traditional division of electronics into subject-matter areas 
was more for convenience of the trainer than as an indication of 
an essential structure. 

5. The AUTOMASTS portion of the proficiency test battery ap- 
peared to have definite potentialities as an advanced trainer for 
shipboard use. 

In the opinion of the writer, the efforts of AFPTRC in develop- 
ing and improving simulators, as described by Gagne and others 
(USAF, 1956; Hughes, 1956; and Briggs and Duvall, 1957), were 
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directed towards training while the ONR-USC research began 
with the development of proficiency testing devices. Yet both 
produced apparatus and programs which by 1956 were, in fact, 
teaching machines and teaching machine curriculums. It is very 
curious that both these developments dealt with maintainability 
of electronic systems, thereby testifying to the early awareness 
by distinguished investigators of the applicability of teaching ma- 
chine technology in this field. 

In 1956, Bryan and others summarized their research in elec- 
tronic trouble shooting, which dealt with sequential records of 
trouble shooting behavior, integration of trouble shooting infor- 
mation with analytic approaches, and general statements about 
trouble shooting; sixty-two conclusions were published (Bryan, 
1956). The implications for future activity supported the concern 
of those in the field of electronic systems mentioned earlier. It 
was believed that the problems were difficult but would yield to 
persistent effort, that fundamental units and categories for coding 
behavior needed to be systematized and elaborated, that contin- 
ued contact with trouble shooting in real work situations rather 
than ivory tower environments was mandatory, and finally that 
the systematic framework was all the more important as elec- 
tronics becomes increasingly important and pervasive in its mili- 
tary and civilian applications. The personnel problems would 
grow in scope and urgency. 

In 1959, Bryan and Schuster reported on research dealing with 
training technicians to efficiently trouble shoot a complex elec- 
tronic system when cookbook procedures had failed (Bryan and 
Schuster, 1959). The experimental design was directed to com- 
pare three magnitudes of structuring: no assistance, guidance 
on steps to take, complete guidance. In the no assistance part, 
learners were required to work through printed Loran trouble 
shooting problems without assistance. In the guidance on steps 
to take part, a specially constructed device called OST (Optimal 
Sequence Trainer) was used to acknowledge correct decisions 
and prompt; it was used together with an information book. It 
had a feature of special interest: if a decision was not forthcom- 
ing in seven minutes, a prompt would be given. This particular 
timed technique has been considered for purposes other than 
maintenance training (Silvern, 1961 and Silvern, 1957). In the 
complete guidance part, learners were required to conform rig- 
idly to a preordained path. The implications were revealing: in 
developing supplemental trouble shooting training procedures, it 
appeared that excellent use could be made of a kind of teaching 
machine since results indicated a properly constructed device 
could greatly improve training. Using the principles developed 
from the experimental evidence, the authors concluded that the 
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Navy would be able to fill a real gap in its present trouble shoot- 
ing training programs by using teaching machines. 

In September I960, addressing the Human Factors Society, 
Rigney stated that it was not likely in the immediate future that 
unscheduled maintenance could be designed out of equipment or 
that fault location could be completely automated. He and his col- 
leagues believed that the most important implication of this and 
related research conducted by others is for the revision of pres- 
ent-day procedures in electronics training schools. The chief 
advantage of the teaching machine was that it would provide a way 
of giving learners a tremendous amount of guided experience in 
a short time by relatively inexpensive means. It seemed to the 
USC group that it was not so much a question of the superiority 
of the machine instruction over traditional classroom methods; 
rather it was a question of giving this training in electronics 
maintenance at all (Rigney, et al., 1960). 

MAINTAINER'S BEHAVIOR AND TRAINING SOLUTIONS 

The Air Force, like the Navy, was aware of the need for de- 
tailed knowledge of electronic system maintenance behavior moti- 
vated by similar reasons. For a number of years the American 
Institute for Research had served a similar function for the Air 
Research and Development Command at the University of South- 
ern California. The work of Miller, Folley, and Altman, intro- 
duced earlier, at AIR, when coupled with the research and de- 
velopment at AFPTRC and WADD as evidenced by the work of 
Briggs, Gagne, Besnard and many others who deserve mention 
for their contributions, represents a magnificent contribution to 
behavioral science in general and maintenance training in partic- 
ular. Unlike the research at USC, the AFPTRC effort was simu- 
lator or training device -oriented. The Subject Matter Trainer, 
E-4 Fire Control System Simulator, E-4 Fire Control System 
Trouble Shooting Trainer, MG-10 Radar Procedural Trainer and 
the MG-10 Computer Procedural and Trouble Shooting Trainer, 
were no longer audio -visual aids or perhaps not even simulators 
as they began to deviate from instructor-centered to learner - 
centered devices. This transition to teaching machine technology 
began very unobtrusively about 1945 and was virtually complete 
when the Subject Matter Trainer and the two MG-10 devices were 
produced by Briggs, circa 1955-57. These were described by the 
author as the pseudo -simulator and more recently by Levine and 
the author as one form of the teaching machine growing from the 
training devices continuum (Silvern, 1957; Levine and Silvern, 
1960; and Levine and Silvern, 1961). There is no question that the 
criteria for teaching machine technology apply to these AFPTRC 
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devices regardless of the names by which they may have been 
identified. 

In August 1959, Briggs summarized his long, persistent and 
productive study of training devices and teaching machines in 
electronic systems maintenance (Briggs, 1959a). He stated that 
the need is crucial to analyze complex skills to establish the 
knowledge components and it was only recently that maintenance 
had become sufficiently demanding to justify the same magnitude 
of research effort which for years had been directed to pilot 
training. He felt that curriculum development had to be improved 
in terms of proficiency outcomes. Briggs saw in 1959 the need 
for verbal or classroom teaching machines and maintenance 
teaching machines, i.e., certain machines with a great usefulness 
for classroom type knowledge and others obviously able to more 
effectively simulate equipment yet retain all the criteria for 
teaching machine technology. He also was aware that engineering 
design could reduce maintainability skills requirements to a point 
where, in some systems, very little practice of maintenance dur- 
ing training would be required. He further described the two dif- 
ferent kinds of teaching machines as the educational kind of ma- 
chines for information acquisition and concept development, while 
special simulator teaching machines may provide practice in in- 
tegration of job knowledge into job task performance (Briggs, 
1959b). Briggs believed that these maintenance teaching ma- 
chines were expensive but did provide learning the actual motor 
performances which he felt were the criterion behaviors and the 
end products of the intellectual activity of the trainee. And he 
was not at all convinced that results obtained in training opera- 
tors contained the answers to the questions raised for maintainer 
training programs. 

Others investigating the use of trouble shooting training devices 
for the Air Force, which were obviously forerunners of the teach- 
ing machine of the maintenance variety, were encouraged by the 
outcomes. Tucker, in using a K-system trouble shooting trainer 
with decreasing or fading guidance by an instructor, concluded 
that it was a "reliable, practical and effective substitute for the 

actual equipment (overcoming objections) that trouble shooting 

can be learned only by experience on the job, that the mechanics 
must be trained on actual equipment that is in excellent operating 
condition, and that a very considerable amount of training time is 
required" (Tucker, 1955). Newman and Highland had concluded, in 
an investigation of the effectiveness of instructional materials 
and methods which would minimize the role of the instructor in 
teaching electronics, that the use of aural -visual devices such as 
the recorder -slide combination would prove useful particularly 
if the materials could be adapted to the needs of the learners by 
preparing several different levels of the same lesson (Newman 
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and Highland, 1956). Stave observed that as Air Force equipment 
became more complex, the training devices also increased in 
complexity with a tendency to provide progressively more auto- 
matic features (Stave, 1960). In tracing the evolution of military 
training devices, he pointed out that various methods were evolved 
to lighten the instructor's task; one method was automatic re- 
cording of critical student responses this was, in his opinion, 

the beginning of the present trend toward automaticity in training 
devices. His seven-step evolution is based upon a human scoring- 
machine scoring discretum. 

Not all investigations or investigators in electronic systems 
maintenance have concluded that teaching machines or even 
simulators hold the final solution. Folley examined the work of 
Smith in which documents were used for trouble shooting (Folley, 
1959a, Folley, 1959b). The technicians had virtually no training 
and only nominal aptitude. A specially prepared job aid in docu- 
ment form was successfully used to trouble shoot. He reported 
that the trouble shooting tree approach was being scrutinized and 
referred to Miller's 1953 anticipatory statement that written job 
instructions might replace training (Miller, 1953). It appeared 
to Folley that all the facts that have been learned on teaching ma- 
chines can now be applied to the design of textbooks that include 
the learning process sequence but he was also aware of the limi- 
tations in maintenance where the acquisition of skills requiring 
dynamic displays and part-task perceptual-motor skills was nec- 
essary. 

The Army, too, is aware of the growing complexity of its elec- 
tronics inventory. Smith recently declared that HumRRO studies 
have shown the value of the functional context method in training 
maintenance technicians (although he does not further identify 
maintenance in terms of the kinds of systems) which consists of 
job -oriented context, graded series of job-related tasks, and a 
whole-to-part sequence (Smith, 1960). He stated that the greatest 
importance of automated techniques in training deals with the pos- 
sibility of applying basic principles for efficient learning. Rupe 
presents, in 1960, the same fundamental problems of complexity, 
length of maintainer's service, predicting training requirements, 
obtaining job information, as were identified earlier in 1955, ad- 
mitting that these were not particularly novel or startling (Rupe, 
1960). His argument is basically that a detailed task and skill 
analysis of a total man-weapon system (both personnel and hard- 
ware subsystems) could be made, reduced to coded form, stored 
in data processing form, made available for modification infor- 
mation and used for many purposes, but the most challenging re- 
quirement will be that for identifying the information essential 
for formal training. Whitmore was concerned with job behavior 
and thought there were many best guesses being made which had 
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to be studied and went on to describe the possibilities inherent 
in teaching machines (Whitmore, 1960). Mager believed, on the 
basis of his research, that while it is possible to impart many 
of the skills related to electronic maintenance, this could not be 
accomplished with a single programing method or with simple 
teaching machines (Mager, 1959). He saw a need for more so- 
phisticated machinery for teaching the skills associated with the 
maintenance of complex electronic equipment. On the other hand, 
Kanner concluded that the area of automated teaching devices is 
one where scepticism should prevail until more evidence and ex- 
perience have been accumulated (Kanner, 1960). 

A SYSTEMATIC APPROACH TO TEACHING MACHINES 
FOR MAINTENANCE TRAINING 

It is not the intent of this paper to provide detailed research 
findings or describe the wide variety of mechanisms known as 
teaching machines, this has been done. In addition, there is little 
to be accomplished in discussing the various methods of develop- 
ing teaching machine lesson plans, the practical value in trans- 
ferring experimental conclusions determined from teaching five- 
year-old children molecular science in the first grade to training 
30 -year -old 7 -level Air Force technicians in trouble shooting the 
MA-11 Digital Computer Subsystems, etc. The fundamental issue 
the writer raises is: can teaching machines be used to train 
maintainers in performing preventative and corrective mainte- 
nance of complex electronic systems? Questions as to the superi- 
ority or inferiority of a machine, the cost of obtaining and main- 
taining a machine, the magnitude of human acceptance of or re- 
sistance to a machine, the logistics all of these will have 

uniquely distinct answers in different frames of reference. 

The weight of evidence appears to suggest strongly that teach- 
ing machines can train humans to proficiently maintain complex 
electronic systems. In order to achieve this objective within rea- 
sonable limits, practitioners should consider the guide points de- 
scribed in the following paragraphs. 

Maintainability Design at the Engineering Stage 

Engineering and physical science curriculums in colleges and 
universities generally avoid providing detailed instruction which 
would result in a strong maintenance design philosophy. As insti- 
tutions of higher education tend to stress fundamentals, even in 
graduate schools because there are so many new fundamentals, 
they depend upon industry to train in the applications of these 
principles. Industry has assumed this role and, through formal 
training programs or supervised experience, attempts to develop 
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such philosophies. Quite often, because of rapid expansion of the 
organization, the supervision may be provided by an engineer 
with only several more years of experience than his subordinates. 
With this pooling of ignorance, the maintenance design philosophy 
can be pretty weird. No, a philosophy to be good should come 
from the systems -oriented engineers with experience these are 
usually in the engineering management layer. But finding the man 
there is not the end, it is only the beginning, since engineers are 
notoriously poor communicators and changing the man to manager 
does not magically make him one. One can conclude therefore 
that sound maintainability design must be planned and specified 
as an integral and fundamental engineering requirement irre- 
spective of engineering management competence. In fact, this 
competence must be acquired, if necessary, to satisfy the cus- 
tomer's specifications. The technical specialties most familiar 
with maintainability problems should contribute to drafting the 
specifications and approving designs. Engineering psychologists, 
behavioral engineers, technical training directors, and other hy- 
brids who have been maintenance oriented are in this category 
of specialization (Holland, 1959). Every effort must be made to 
design and manufacture a system which minimizes the need for 
corrective maintenance and has nominal preventative mainte- 
nance. Systems which by design are difficult to maintain will 
present equally difficult job behavior problems which, in turn, 
will create nearly insurmountable teaching machine curriculum 
problems. Training is not always the real solution quite often 
better results can be obtained by early engineering decisions or 
by early management decisions rather than by after -the -fact 
training programs. 

Job Information and Behavioral Patterns 

When electronics consisted mainly of communications -type 
equipments, that part of maintenance called trouble shooting was 
pretty straightforward or, as some have said, from the middle to 
one end! From the fantastic variety of new components, trick 
circuits, microminiaturized packages, modular designs and 
strange environmental conditions of temperature, pressure, hu- 
midity, and radioactivity, it is with difficulty at times that one 
can trace a signal or even locate a stored bit. It is probable that 
patterns of trouble shooting behavior which apply to signal-flow 
type circuits might not apply to storage and pulse type circuits 
typified by computer logical design. This suggests that behav- 
ioral analyses might be required for each total system and the 
results might reveal fundamental differences in behavioral pat- 
terns. If all electronic systems elicitedthe same general trouble- 
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shooting behavioral patterns then there would be a general formu- 
larization, but if special systems revealed different patterns then 
a family of formulas would be created. This outcome should have 
a profound influence on the selection of the general -purpose or 
special-purpose teaching machine and the design of the latter. 

General -Purpose and Special-Purpose Teaching Machines 

Briggs visualized two kinds of teaching machines: verbal or 
classroom and simulator or maintenance; and he is completely 
correct in identifying them separately (Briggs, 1959a and Briggs, 
1959b). The difficulty in which the teaching machine field finds 
itself is the absence of a suitable vocabulary with which it can 
communicate. This matter is important because Briggs' concept 
must be defined in such a manner that it can be applied to fields 
other than maintenance training if the concept is what it appears 
to be. The author identified the general -purpose teachine ma- 
chine as one by which a variety of different subjects may be 
taught, as versatile and flexible to satisfy multi-directional ob- 
jectives (of typical correspondence programs), and the special- 
purpose teaching machine which included simulators which were 
fixed-programed so as to treat only one kind of subject matter or 
by which only one specified subject may be taught (Silvern, 1961a) 
and Silvern, 1961b). The author also classified general -purpose 
teaching machines as consisting of visual (manual, mechanical, 
electromechanical, electro -optical); aural (electrical); aural- 
visual (electro-optical); and olfactory, gustatory, tactile. Levine 
and the author further described the general purpose as one which 
will permit a fixed variety of methods with a nearly unlimited 
variety of contents; in which methods were equated to psychologi- 
cal principle and contents consisted of subject-matter (Levine 
and Silvern, 1960 and Levine and Silvern, 1961). Levine said that 
he understood the general-purpose teaching machine to be one 
designed for multiple subject matter, in which the displays and 
types of student responses were universal in nature and appli- 
cable to an infinite number of related or unrelated subjects; in- 
formation stored in the device might be easily removed or inserted 
(Levine, 1960). Although the controlling mechanism of any general- 
purpose teaching machine is unique to the device itself, most any 
content can be geared to be compatible with this mechanism. As 
far as the device is concerned, he said, it is designed independently 
of what it will teach. In Levine's analysis, the special-purpose 
teaching machine was designed so that only one specific body of 
knowledge is learned. It cannot be used for any other purpose 
certain unique content, requiring special indicators, opera- 
tions and/or responses, was to be taught. If any feature, be it the 
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method of responding and/or the type of indicator and/or content, 
etc., is unique, the machine would be considered as special-pur- 
pose. To change the content would mean to actually physically re- 
design the machine itself. Because the design and use is for one 
specific purpose, these are called special-purpose teaching ma- 
chines (Levine, 1960). 

With these definitions, Briggs' maintenance teaching machine 
and simulator teaching machine would fall under special -purpose 
teaching machines and logically this should yield an analysis simi- 
lar to the one for general -purpose. However, it is at this point 
that the sparks begin to fly, since there is little agreement on the 
nouns: simulator, trainer, et al. Perhaps this matter should be 
deferred for another time, but it is obvious the terms simulator 
and simulation have gotten out of hand and need clarification if 
we are to proceed with this analysis. 

The writer has been studying the possibility of designing a 
special -purpose teaching machine driven by a general -purpose 
digital computer of standard design and configuration a shelf 
item, so to speak. Different special -purpose teaching machines 
driven by the same computer would be the parallel case of the 
general -purpose teaching machine in groups of, let us say, twenty 
driven by a central, general -purpose computer as mentioned by 
Weimer, Ramo, Silvern, and several others and which is under in- 
vestigation by Coulson and Silberman at System Development 
Corporation (Silvern, 1961a). The computer control is a common 
practice in engineering and human factors laboratories where 
analog and digital simulation is required. 

The job analysis and behavioral studies would establish the var- 
ious knowledge and manipulative activities which are directly, in- 
directly, or generally related to the end performance objectives 
or unrelated to them and therefore confusion-generating. Together 
with job performance standards, this information should result in 
selection decisions with respect to the types of teaching machines 
best suited for the maintenance learning environment. The locale 
for such a system, which is some years away, would probably be 
a technical school. With the rapid acceptance of general -purpose 
digital computers having high-speed, time-sharing capabilities, 
only the computer input-output coupling, the teaching machines, 
and the computer program would be needed to add to an existing 
computer in use for other than learning purposes, thereby reduc- 
ing capital expenditures. 

Those who work with, think about, and write about teaching ma- 
chine technology often make statements which while general in 
scope really should be limited to specific areas. Eight such areas 
of application for teaching machines have been identified as fol- 
lows (Levine and Silvern, 1960 and Levine and Silvern, 1961): 
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1. Business and industry. 

2. Government (non-military). 

3. Military. 

4. Public education (K to 12th year). 

5. Higher education. 

6. Adult and special education. 

7. Education and training assistance to foreign countries. 

8. Home and family. 

This article deals almost exclusively with military with some 
small application to business and industry and no reference to 
any of the others. In this area, the system described can develop 
quickly due to need. This system concept, however, will not be 
accepted as rapidly in other areas, as has been pointed out (Sil- 
vern, 1961c), due to the unique characteristics of the areas. 

Curriculum Development 

The most exciting part of teaching machine technology is, of 
course, the development of the lesson itself. Silvern and Levine 
have described general methods of developing teaching machine 
lesson plans for general -purpose teaching machines which essen- 
tially follow successful lesson planning practices used over a long 
period of years in technical training environments. The steps 
generally involve the following: 

1. establishing job performance criteria; performance stand- 
ards dictate the design of the training program and are established 
by analysis. 




2. producing a job analysis; performance standards are de- 
veloped into job analysis by the process of synthesis. 



Job 
analysis 




Job 
performance 
standards 





3. creating a course outline from the job analysis by analysis. 



Job 
analysis 




Course 
outline 
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4. creating lesson plans next by analysis; consisting of main 
facts and teaching points. 
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5. relating human instruction curriculum to machine instruc- 
tion curriculum. 
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In developing a teaching machine curriculum for a course which 
has not been instructed by a human instructor, it is possible to 
simplify the lesson plan step preceding the teaching machine les- 
son plan. 

This applies to general-purpose teaching machine curriculum 
developmentthe classroom situation, usually, as stated by 
Briggs (1959a and 1959b). In turning to the special-purpose 
teaching machine, it is evident that behavioral studies, such as 
those conducted by the Electronics Personnel Research Group at 
the University of Southern California, the American Institute for 
Research, and other cited earlier, will contribute heavily to the 
job performance standards and the job analysis. Generally, how- 
ever, the main steps in moving from end performance objectives 
to job analysis, etc., should be unchanged except that the lesson 
plan will be controlled by the extent of whole -task, part-task 
philosophies utilized in the task identification. And, of course, 
the information displays, such as screen projection (visual) and 
taped voice (aural) now used in the general -purpose teaching ma- 
chine, will probably appear as visual displays (scope, meter, 
neon indicator, switch or dial position, screen, et al) in the spe- 
cial-purpose teaching machine. One can think of an application 
of branching techniques with aural display in which the wrong de- 
cision on the special -purpose machine by the learner is not only 
identified to him, and scored, but also his error is aurally de- 
scribed although he may not be prompted; here too could be an 
application of the principle of fading. At this point, the writer 
perceives that he is dreaming as an engineer interested in gadge- 
try rather than as a behavioral scientist or training specialist 
armed with experimental evidence justifying these clever feed- 
back schemes! 
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Purposes 
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A teaching machine may be broadly defined as a device the de- 
sign of which is based on the laws of learning and with which, 
through interaction between it and a student, the student learns 
what is prescribed for the machine to teach. This article is par- 
ticularly concerned with the "interaction* ' of the machine and 
the student. How should the student respond to the machine, and 
how should the machine respond to the student? 

The main objective of teaching-machine instruction is to have 
the student learn what is prescribed for the machine to teach. 
(This is the essence and sole purpose of any learning situation.) 
It follows, therefore, that the reaction of student to machine and 
machine to student should be such as to best facilitate the learn- 
ing process. In a given learning situation, if the student will learn 
better by being presented a picture, or a knob setting, or a nut 
and bolt, this should be the type of presentation he receives, all 
other things being equal. 

By the same token, the response of the student should be se- 
lected to optimize learning, all other things being equal. The re- 
sponse he makes might be writing an answer, pressing a button, 
adjusting a knob, typing an answer, or an infinite number of other 
responses. The key to the interaction of student and machine lies 
in the selection of presentations and responses, for a given situa- 
tion, that best facilitate learning, all other things being equal. 

In describing optimum interaction between man and machine 
above, the phrase "all other things being equal" was repeated 
several times. Unfortunately, all other things are never equal. 
Thus, the type of presentation and student response selected 
should be the result of an evaluation of all factors and their 
consequences in a learning situation. For example, in teaching 
a computer card-punch operation, the student might learn better 
if his response is made on a card-punch rather than a simulated 
board. The slight expected increase in learning, however, usually 
does not warrant the cost of an actual card-punch. 

TWO TYPES OF TEACHING MACHINES 

In many courses, we find that the type of interaction between 
man and machine is not critically meaningful. A given type of 
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response, such as pressing a multiple choice button or writing 
an answer, can be used for many different courses. This flexi- 
bility enables one machine to teach a multitude of courses. These 
machines, with a general type of interaction, are called general- 
purpose teaching machines. 

There are learning situations, however, in which the type of 
man-machine interaction is critical. Other factors make it fea- 
sible to use a machine that is capable of teaching only one body 
of knowledge. This machine has little flexibility, but the learning 
situation is important enough to forego this capability. These 
machines, with a special type of interaction, are called special- 
purpose teaching machines. 

General-Purpose Teaching Machines 

Many teaching-machine experiments have indicated that the 
essence of success lies in the information and/or questions and 
the sequencing of them, not in the machine itself. In part, the 
reason for this situation, which maximizes the importance of the 
content, is the fact that most machines provide a versatile, flexi- 
ble, satisfactory mechanism by which the content can interact 
with the student. Although the mechanism is important, it is the 
content and the psychological method that provides the greatest 
effect on the learner. 

The general -purpose teaching machine is one designed for 
multiple subject matter. The displays and types of student re- 
sponses are universal in nature and applicable to an infinite num- 
ber of related or unrelated subjects. Information stored in the 
device can be easily removed or inserted; the displays usually 
are slide or film -strip projections or paper appearing at an open 
window, but audio presentations also can be used. The majority 
of such machines require one of two types of responses; the mul- 
tiple choice or the constructed response. Similarities in presen- 
tations and responses reduce the difficulty in learning the opera- 
tion of new machines. 

Although the controlling mechanism of any general-purpose 
teaching machine is unique to the device itself, almost any con- 
tent can be geared to be compatible with this mechanism. Thus, 
as far as the device itself is concerned, it is designed indepen- 
dently of what it will teach. In addition, these machines usually 
are small, light, and portable. 

The general -purpose teaching machine has the following five 
basic functions: input, output, comparison, control, and storage. 
A block diagram of the machine would look like that shown in 
Fig. 6-4-1 and would be described as follows: 

a. The input unit. This unit accepts the student's responses 
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and presents them in a form compatible with the requirements 
of the comparator. The responses may be in the form of buttons 
to be pressed, written-in answers, or combinations of these. 

b. The output unit. This unit presents information and ques- 
tions, an indication of correct response, and a record of the stu- 
dent's progress. The presentation may be projected on a screen 
or written on a piece of paper. It may be audio, visual, or audio- 
visual. 

c. The comparator unit. This unit compares the student re- 
sponse input with the correct response from the storage unit and 
determines if the answer is right or wrong. 

d. The control unit. This unit determines the next question, 
records on a permanent record, and provides reinforcement 
through lights, etc. 

e. The storage unit. This unit stores the lesson plan and the 
permanent record. 

Special-Purpose Teaching Machines 

The special -purpose teaching machine is designed so that only 
one specific body of knowledge can be learned. It cannot be used 
for any other purpose. One may ask the reason for such an in- 
flexible design. The answer is that certain unique content that 
requires special indicators , operation, and/or responses must be 
taught on a special -purpose machine. If any feature, be it the 
method of responding, and/or the type of indicator., and/or con- 
tent, etc., is unique, the machine is considered a special-purpose 
one. The other features may vary. 

An example of a special purpose teaching machine is one that 
teaches radioteletype operation. The probable uniqueness of the 
input keyboard would designate this as applicable only to the 
teaching of radioteletype operations. 
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Usually, special -purpose teaching machines are used when the 
learning task is important or costly enough to make it financially 
profitable to go to the expense of manufacturing special machines 
for instruction. In certain situations a special -purpose machine 
might replace the actual equipment in a learning situation. In this 
case, although the teaching machine is expensive, it may very 
well be a lot less expensive than the equipment that it replaces. 

THE FUTURE 

As is usually the case, after certain categories are established, 
the tendency is to join them in special combinations. The future 
probably will reveal general -purpose machines with special -pur- 
pose accessories (for example, a basic general -purpose course 
in radar fundamentals with accessories for developing certain 
trouble shooting techniques for a specific radar set). In addition, 
the growth of simulators probably will benefit by recent develop- 
ments in teaching -machine technology, and simulators will be de- 
veloped with teaching-machine capabilities. 



SECTION 1. Programing Agencies 



This section is concerned with the programing efforts and 
methods of three different programing organizations. Each 
article is written from a different point of view and empha- 
sizes the various problem areas encountered by the authors. 

The first article examines programed instruction as a train- 
ing tool for mail-order clerks and telephone-relay adjusters. 
Focus is placed on an analysis of the conventional teaching 
methods used for mail-order-clerk training, with a discussion 
of particular training problems that could be relieved by pro- 
gramed instruction. Details of program construction and prob- 
lems of program utilization within the framework of the total 
training system are considered. A cost analysis is presented 
that estimates the savings resulting from reduced training 
time for instructors and personnel and from increased task 
efficiency. 

The second article describes research by Schering Corp. di- 
rected toward testing the effectiveness of a programed course 
for pharmaceutical-representative training. Two groups of 
trainees are compared. One group received conventional class- 
room lectures that were augmented by a standard textbook. 
The second group worked at home with a programed textbook 
that covered the same material. Results indicated that the pro- 
gramed -textbook group obtained superior grades on a company- 
administered examination of the course material. 

The third article describes the principles of mathetics,a sys- 
tem of developing instructional material that differs from both 
the standard constructed-response programing technique and 
the multiple-choice-branching technique. The authors provide 
a detailed description of their method, and explain the special 
terminology that is used. In addition, an extensive table is 
presented that compares the mathetical approach to the more 
well -known, standard programing methods. 
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Programed Instruction in Business and 
Industry 

By ALBERT E. HICKEY 
Entelek Corporation 



The quickening pace of technological and social change means 
more attention to learning the principal process of adaptation 
to change. In educational and military establishments, great 
strides have been made toward more efficient learning partic- 
ularly in their adoption of teaching machines and programed in- 
struction. Programed instruction also has been applied to indus- 
trial and business training, with improved quality and reduced 
training costs. Programed instruction cuts training costs by 
reducing the average number of training hours required and by 
decreasing the requirement for skilled instructors or training 
supervisors. In addition, the cost of programed instruction, 
when measured against the potential saving, usually places the 
training project using this technique well up on the list of prior- 
ities given to industrial improvements. 

Before getting into case histories that demonstrate the truth 
of the above statements, we will show how programed instruc- 
tion provides a bonus by encouraging a rigorous analysis of the 
job itself, its role in the system, and the cost of training for it, 
and how, to be assured of a satisfactory outcome, the training 
director must attend, not only to the obvious tactics of reinforce- 
ment (the missing words and immediate knowledge of results), 
but also to the strategy of ordering the subject matter in the 
programed sequence. 

MANUAL vs. VERBAL SKILLS 

The teaching machine is a rather specialized device reflecting 
its origin in the laboratory of Dr. B. F. Skinner. According to 
Dr. Skinner, complex behavior is shaped in an evolutionary man- 
ner by refining an organism's emitted behavior in many small 
steps a process easily demonstrated in simple organisms such 
as the pigeon. In humans, Dr. Skinner points to the shaping of 
verbal behavior, especially the acquisition of vocabulary, as a 
representative process. 

Reflecting his interest in verbal behavior, Dr. Skinner's pro- 
totype teaching machine is uniquely adapted to verbal learning, 
that is, the learning of mathematics, languages, and other skills 
that require an oral or written response by the student. This 
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means that it also is directly applicable to a large variety of 
white-collar skills in business and industry. The machine, how- 
ever, cannot be used to teach manual skills, which are often of 
particular importance in industry. 

For verbal skill, comprehension of the underlying concepts, 
say of French grammar, is evident through the student's verbal 
expression. For a manual skill, such as aircraft control or the 
adjustment of telephone relays, comprehension of the underlying 
concepts, say of aerodynamic response in the first case and 
mechanical function in the second, is evident in the manual op- 
erations of the trainee. To exercise a verbal skill, the student 
needs only a pencil and paper to write out his verbal response 
or magnetic tape on which to record it. A manual skill, on the 
other hand, must be exercised on the actual equipment or a 
reasonable facsimile thereof. Driving, for example, is taught 
in an automobile. Flying, for the sake of economy and safety, 
is often taught in aircraft simulators such as the familiar Link 
Trainer. 

Machines for teaching manual skills, therefore, must be more 
elaborate, specialized, and expensive than the relatively simple 
teaching machines or programed textbooks that are suited to 
teaching verbal skills. Machines for teaching manual skills 
are much more in the spirit of the simulators used by the mili- 
tary. Actually, simulators already embody some of the impor- 
tant characteristics of teaching machines. For example, they 
give the instructor considerable control over the trainee's be- 
havior, and the instructor can program the input to the trainee 
and provide appropriate verbal reinforcement. (The pilot in a 
Link trainer, for example, can be required to "land" many 
times in succession without having to "take-off.") In addition, 
malfunctions can be introduced into the equipment by the in- 
structor. 

Generally, the instructional program for an aircraft simula- 
tor is devised on the spur of the moment by the instructor based 
on his long training experience and his knowledge of the fledgling 
pilot's individual training requirements. This procedure often 
is very effective, but it requires gifted instructors of long ex- 
perience working with individual trainees. Furthermore, the 
machine itself seldom measures the necessary performance 
characteristics or compares such results with criteria for 
"good" performance at least not in time to be of maximum 
use to the student. This defect is not necessarily due to a lack 
of engineering ingenuity or funds. Rather, it is due to a failure, 
or the inability, to specify in quantitative, unequivocal terms 
what constitutes good performance. The determination to de- 
velop a programed-instruction sequence with reinforcement at 
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optimum intervals often leads to the necessary definition of 
criteria and to the development of useful scores and scoring 
devices. 

TACTICS AND STRATEGY IN PROGRAMING 

The obvious features of a teaching-machine program (or pro- 
gramed textbook) are the short items, missing words, frequent 
repetition of key responses, and immediate knowledge of results. 
These details, however, which largely implement the principle 
of reinforcement, might be so obvious that they obscure the 
more diffuse, but equally important, characteristics of an opti- 
mum learning sequence. The order in which topics are covered 
in the sequence is determined by a consideration of such princi- 
ples of learning as positive and negative transfer, stimulus and 
response generalization, stimulus discrimination, response dif- 
ferentiation, and span of retention. 

Although these principles certainly can be applied without the 
step-like reinforcement procedures introduced by the Skinner 
machine, the orderly presentation of stimuli makes the larger 
strategy easier to implement. One might say that the careful 
provision for reinforcement provides the motive power for the 
learning process, whereas the consideration of transfer, reten- 
tion, etc. determine the route the student or trainee is to take 
through the subject matter. It is in the planning of this route 
that the psychologist makes his major contribution to the de- 
sign of a programed-instruction sequence. This is true whether 
the skills to be learned are verbal or manual. The route is, 
however, much more important in training a complex clerical 
procedure than in the rote learning of nomenclature. 

The pitfall that awaits the unsuspecting training director is 
that, seeing only the blanks in the items, he will fail to appre- 
ciate the strategic considerations in the design of the program. 
Strategy often is more important in an industrial training pro- 
gram than it is in an instructional sequence for academic sub- 
ject matter. Most academic subjects already follow a more or 
less traditional curriculum and syllabus. Generally, the peda- 
gogical sequence of the syllabus has been well tried in the 
classroom and has been reorganized many times to make it 
more effective. The industrial training director, on the other 
hand, often must take as his point of departure a procedures 
manual written by industrial system engineers, and the necess- 
ity for reorganizing the content of the manual into a training 
sequence is apparent. 
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BUSINESS AND INDUSTRIAL TRAINING 

Business and industry present a complete spectrum of skills 
ranging from manual to verbal. Three applications one recent, 
one current, and one projected serve to demonstrate the appli- 
cation of programed instruction to white-collar group training 
on predominantly verbal tasks and to blue-collar apprentice 
training on a manual task. Cases A and B describe programed 
instruction for package -billing clerks and credit-accounting 
correspondents. Case C is an analysis of training costs and 
projected savings in programed instruction for telephone relay 
adjusters. 

Case A Training Package -Billing Clerks 

Several characteristics of the mail-order business make it a 
good proving ground for programed instruction. It has the large 
annual turnover characteristic of retail business, and most of 
the training is concentrated in the fall just prior to the Christ- 
mas rush. Furthermore, many of the tasks to be taught are of 
a clerical nature and do not require specialized training equip- 
ment. 

The task chosen for a pilot study at Spiegel, Inc. was package 
billingthe final clerical operation in the preparation of a pack- 
age to be mailed to a customer. Spiegel hires about 500 girls 
for this job each fall. These package billers do the following: 

a. Sort out the sales slips. 

b. Note the cancellations and substitutions that have been 
made in filling the order. 

c. Total the merchandise price. 

d. Calculate a variety of sales and excise taxes, coupon allow- 
ance, postage, etc. 

Conventional Instruction. Spiegel grouped girls to be trained 
as package billers in classes of 15 girls each. Up to four new 
classes were begun each week during the height of the training 
season. The ten days of this conventional group training were 
divided between classroom instruction and practice in billing of 
packages a on the line." 

A total of 40 hr were devoted to classroom instruction. During 
the first several hours in class, the trainees learned to add using 
the Comptometer. Then they worked through a graded sequence 
of 75 sample billings. Some of the sample billings introduced 
new clerical forms or routines; others provided an opportunity 
for practice. The instructor explained each new point and made 
occasional blackboard notes , She waited for every trainee to complete 
each sample billing before moving on to the next one. Short answer 
tests were given frequently, were scored, and then reviewed in class. 
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When they had completed 32 hr of classroom instruction, the 
trainees reported to the billing line and billed sample packages 
under the surveillance of their instructor. The instructor checked 
100 % of their work and tallied the number of packages processed 
and the number and kind of errors. The systematic collection of 
these performance data subsequently made it possible to make a 
very exact quantitative evaluation of the instruction. 

After 20 hr "on the line," the group returned to the classroom 
for 8 hr of additional instruction in Comptometer subtraction and 
multiplication and "cash" billing. The "cash" training consisted 
of 55 sample billings. Finally, after one more day of billing "on 
the line" under 100% supervision, the girls were given regular 
work assignments under the usual 4 % quality- control check. 

Programed Instruction. As the first step in designing a new 
course of programed instruction, the consultants took the two- 
week course for package billers. They did not emerge as very 
skillful billers, but they did compile a detailed description of the 
biller's task. Furthermore, they had a chance to look at training 
from the point of view of the trainee not the billing expert. 

Based on their classroom notes and on established principles 
of learning, the training sequence was reorganized. In the new 
instructional sequence, all billing operations that required the 
same responses were grouped together, and the billers were 
trained to recognize the cues associated with each of these 
general response categories. After each general response cate- 
gory was established through practice, the billers were trained 
to make finer discriminations. 

Several techniques were used to expedite the recognition of 
cues. For example, when a new clerical form was introduced, 
it contrasted with other forms with which the trainee was al- 
ready familiar. New symbols, designed to trigger specific 
clerical routines, were heavily outlined at first. The outline 
then was gradually diminished over successive exposures until 
the trainee could detect the cue without this crutch. 

As soon as the trainee could recognize all the cues that sig- 
nify a particular response or clerical procedure, the program 
was switched to teaching the procedure itself. First, the trainee 
was required to trace the correct response in the correct loca- 
tion. Then she marked it herself with progressively less assist- 
ance from the program. Throughout the program, cue-response 
combinations previously learned were rehearsed in new contexts 
to increase the generality of the response. As learning progressed, 
simple cue-response combinations were put together in more com- 
plex sequences leading up to the complete package -billing per- 
formance. 

Programed Workbooks, Programed workbooks rather than 
teaching machines were used. Although package billing involves 
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many verbal responses, particularly the writing of numbers, an 
important part of the job is to write the numbers in the correct 
spot on hundreds of more or less complex forms. Each of the 
forms, therefore, must be presented many times and in such a 
way that the biller actually can write in the results of her com- 
putations. Although this requirement conceivably might be met 
by a very elaborate teaching machine, it is clear that the ma- 
chine would not be economically feasible. 

In all, a series of eleven programed workbooks were used by 
each trainee. Seven of the books cover the billing procedures 
themselves, two deal with the Comptometer, one with tax calcu- 
lations, and one provides refresher training. The pages of the 
loose-leaf workbooks are of heavy-stock paper. On the left-hand 
page, and sometimes at the top of the right-hand page, are the 
clerical forms necessary to introduce a new cue or clerical 
procedure. In the lower half of the right-hand page is a set of 
1-6 instruction cards. The 8x5 cards are hinged at the top and 
overlap like shingles. On the front of each card is a statement 
about the forms. A blank in the statement indicates to the trainee 
that she must write out some important missing information on 
a separate pad. After making her response, the trainee checks 
it against the correct response, which is printed on the back of 
the card. Sometimes, instead of writing her answer on the pad, 
the trainee is directed to make her response on the clerical 
forms attached to the facing page. Tissue overlays protect the 
forms for future use. 

The Experiment. One hundred and twenty package billing 
trainees participated in the evaluation in 1960. They were as- 
signed to eight classes of 15 each. Four of the classes received 
conventional group training. The remaining four classes re- 
ceived their classroom instruction from the programed work- 
books. In the experimental classes, the instructor spent her 
time correcting the quizzes that were integrated with the self- 
instruction pages, exchanging books, and giving assistance to 
individual trainees. As might be expected when each trainee 
sets her own pace, some students were soon one, two, or three 
books ahead of the other students. To avoid confusion, a num- 
bered card at each desk indicated to the instructor which book 
the trainee was working on at a given time. 

The sixty girls that used the programed-instruction method 
required an average of 26 classroom hours to learn package 
billing a saving of 34% over the 40 hr normally allotted in the 
conventional group training. (There was an average difference 
of 10 1/2 hr between the fastest and slowest trainee in each 
class.) This reduction in classroom hours was achieved without 
impairing their performance on the job. 

Further innovations, which could not be tested during the 1960 
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training season, might reduce the persistent 3 % error rate in 
package billing. In 1961, for example, some of the classroom 
time saved during "cash" training in the second week was re- 
invested in a review of some infrequent credit operations taught 
during the first week but not encountered during practice "on 
the line." It is likely that, without practice or review, these 
infrequent operations drop out of the trainee's repertoire shortly 
after they are learned. This would account for a sizeable part of 
the residual error rate in package billing. 

Some Fringe Benefits. A more promising antidote for the resid- 
ual errors doesn't involve training at all, but it does demonstrate 
one of the major incidental benefits of programed instruction 
methodism. During the training experiment, it was discovered 
that 40 % of the package -billing errors resulted in "incorrect 
total postage." Two-thirds of the time, this error was due to a 
failure by the package billers to recognize the 14 different sales - 
slip symbols that tell her additional postage must be figured be- 
cause some of the merchandise ordered by the customer will be 
sent separately. 

In view of the fact that the 14 symbols all meant the same 
thing to the biller (but not to the back-order clerk), it was sug- 
gested that a plus sign be added in the lower right corner of any 
sales slip that calls for additional postage. This procedure 
would obviate the need for recognition training on the 14 cues 
and reduce the "bandwidth" requirement for the package biller. 
Fortunately, the back-order clerk already was detecting the 
additional -postage items for her own purposes, although she did 
not then pass the information along to the package biller; this 
information was, in a sense, thrown away. Because the back- 
order clerks customarily have a very low error rate, they can 
add the plus sign without jeopardizing their own efficiency. 

Incentive Plan. The Spiegel incentive program is a key factor 
in motivation and should be well understood by the employees. 
If the incentive plan is not well understood, or if it is misunder- 
stood, the effect on production and quality of performance could 
be quite different from that hoped for by the industrial-engineer- 
ing department. The complex Spiegel incentive plan is custom- 
arily outlined to the employee in a small brochure. As with 
many other booklets on procedures and benefits, it is doubtful 
that even a majority of the employees read it thoroughly enough 
to understand how their errors and production affect their pay 
envelope. As part of this project, the contents of the Spiegel 
brochure were remodeled as a teaching-machine program. A 
formal study was not made of the employees' reaction to the new 
presentation, but supervisors reacted favorably, and the program 
was adapted to other jobs. 
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Some Administrative Pros and Cons. If an hourly rate of $1.50 
and 100 % overhead are assumed, the 34% saving in the conven- 
tional allowance of 40 classroom hours per trainee yields a po- 
tential saving of $20,400/yr if 500 package billers are trained. 
This, of course, is an average saving; each trainee proceeds at 
his own pace and might finish the course before or after the 
others. 

In 1960 the staggered output created problems for the super- 
visors of the package -billing department. The trainees had to 
be regrouped for practice "on the line," and the time saved by 
individual training in the classroom was lost. To avoid this 
loss, it would be necessary to set up a "training pool" to which 
new employees would be assigned as soon as they were hired. 
They would leave the training pool for regular work assignments 
as soon as they completed the training program. 

]ji addition to a reduction in the average number of classroom 
hours, programed instruction led to more efficient use of skilled 
instructors, particularly if the instructor had a clerical assist- 
ant to carry on the routine functions of distributing and collect- 
ing training materials, correcting quizzes, etc. The availability 
of the clerk permitted the instructor to devote her full time to 
helping the less apt trainees. With a clerk, Spiegel instructors 
could monitor twice the number of trainees they taught when 
using the conventional group-instruction methods. 

Case B Credit-Accounting Correspondents 

In 1961, Spiegel applied programed instruction to another mail- 
order job: that of the credit-accounting correspondent (CAC). 
About 45 girls are trained each year to handle mail containing 
customers' complaints or inquiries about discrepancies, real or 
fancied, in their account balances. The CAC determines whether 
or not an accounting error has been made, initiates corrective 
action if necessary, develops additional information as necessary, 
and placates the customer. 

In conventional training for this job, a supervisor takes about 
40 hr to review the procedures manual with a class of 6-8 train- 
ees. Next, the trainees spend eight weeks handling mail under 
100 % supervision. Spiegel records show that, throughout this 
eight-week period, the trainees handle only 2-3 letters per 
hourabout 15% of standard for the job and that they do not 
approach standard until some months thereafter. 

This slow progress may not be entirely a training problem. 
CAC is not a bonus job, and low motivation might be retarding 
improvement. CAC trainees who usually handle 2-3 letters per 
hour increased their output to ten letters an hour for a 1-hr 
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period while they were being monitored by project personnel. 

Following conventional instruction, the average error rate at 
the beginning of the eight-week period under 100 % supervision 
was about 50 %. This rate improved to about 20 % at the end of 
the eight weeks, but the reduction in errors could be attributed 
to the fact that three out of seven trainees were dropped by the 
end of the period. 

As a case for programed instruction, the CAC job is different 
in many interesting respects from package billing. There is, 
for example, a more uniform training requirement throughout 
the year and smaller classes, which means an easier tolerance 
for the staggered output of individual training. Unlike training 
for package billing, the CAC job is taught from the procedures 
manual, and little use is made of cases, so that there is little 
classroom practice on typical behavior patterns. 

The CAC job also has two widely different components. One 
is the accounting procedure, the other is the simple referral of 
mail to a variety of other Spiegel activities. The latter compon- 
ent is close to what the psychologist calls paired-associates rote 
learning and represents the first economic opportunity to use 
teaching machines for programed instruction at Spiegel. 

Case C Training Telephone Relay Adjusters 

A prominent manufacturer of telephone relays produces up to 
6,000 relays per day by automatic methods. About 120 men and 
women are employed to make the final adjustments on the relays. 
The total manufacturing cost for the average relay is about three 
dollars. The cost of actually making the relay is about two dol- 
lars; the balance of the manufacturing cost, about a dollar per 
relay, is due to the manual operation of adjustment. Any further 
reduction in the overall manufacturing cost will have to be 
achieved through economies in the adjustment operation. 

One of the most promising places to look for savings in relay 
adjustment is in the training of the relay adjusters, about 40 of 
whom are trained each year. Each new adjuster is trained indi- 
vidually by assignment to an experienced adjuster. He receives 
a half day of individual lecture and demonstration on the first 
day and then begins to adjust simple relays under the supervision 
of his experienced "buddy." Under this system, it takes about 
three man-months for the average trainee to reach 70 % of the 
production standard for that department. The standard is based 
on both rate and quality of production. At the end of one year, 
the average trainee has achieved between 85 and 100% of the 
production standard. 

Training Costs. The cost of training an apprentice relay 



PROGRAMED INSTRUCTION IN BUSINESS AND INDUSTRY 291 

adjuster before and after programed instruction is shown by the 
curves in Fig. 7-1-1. To estimate the prospects for savings, we 
assume that, in this department, one man-day with overhead 
costs about $40 and that there are about 250 working days per 
year. The area under each curve represents the value of the 
production of the trainee during the first year. The area above 
each curve is the cost of training in terms of lost production. 




40 60 



250 



Days 



Fig, 7-1-1 Cost of training apprentice relay adjuster before 
and after programed instruction 

Before programed instruction, the cost was $2,000 for one 
trainee or $80,000 for 40. With programed instruction, the time 
required to reach 70 % of the production standard is reduced by 
one-third (40 vs. 60 man-days) as shown, and the cost of training 
is reduced by about 26% or $21,000 for 40 trainees. The esti- 
mated one-third saving in the time required to reach 70 % of the 
production standard is based on the results obtained in training 
the package billers at Spiegel. 

To the cost of training an apprentice relay adjuster should be 
added the cost of supervising the training by the experienced 
adjustor, which is shown in Fig. 7-1-2. In lieu of more exact 
data, we make the conservative assumption that the experienced 
adjustor devotes 50 % of his time to his apprentice at the outset 
of training and that this "nonproductive" supervision decreases 
in proportion to the growth in the trainee's production. 

Before programed instruction, the experienced adjustor sacri- 
ficed about 10% of his productivity, equivalent to $1,000 per year, 
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Fig, 7-1-2 Cost of supervising training of relay adjuster be- 
fore and after programed instruction 

to training. Assuming a one-to-one ratio of training " super- 
visors " to trainees, the total value of production lost from 40 
experienced adjusters is about $40,000 per year. Again basing 
our estimate on the Spiegel study, we assume the requirement 
for trainee supervision can be cut in half by the introduction of 
programed instruction. Thus, the potential saving in super- 
visory time is $20,000. Added to the $21,000 saving in trainee 
production, this maJkes a total estimated saving of $41,000 per 
year (on a current training cost of $120,000 a one-third reduc- 
tion). The saving is further increased if we assume that more 
trainees will complete the instruction program. 

Even more than package billing, relay adjustment requires 
manual as opposed to verbal skills. To "act out" his response, 
the relay adjuster needs a relay, or something like a relay, and 
the appropriate tools. Because it would be impractical to have 
the trainee read the successive instructions in the programed 
sequence, they should be read to him (via magnetic tape) and 
demonstrated graphically with either still or motion pictures. 
This requirement for an audio -visual input to the trainee and 
for a relay simulator on which to perform the necessary man- 
ual operations markedly increases the initial cost of programed 
instruction for manual training over the cost of a teaching ma- 
chine for verbal learning. There is every reason to believe, how- 
ever, that the entire cost of the project, including the initial cost 
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of equipping five relay-adjustor training stations, can be recov- 
ered in a year or less. 

HOW MUCH DOES PROGRAMED INSTRUCTION COST? 

Two factors determine the cost of writing a programed- 
instruction sequence to be substituted for conventional group 
training of a verbal skill. One factor is the number of items 
that will constitute the program. The other factor is the time 
required to prepare each item. The number of classroom hours 
usually devoted to the subject is the best basis for estimating 
the number of items that will be required in a verbal-learning 
program. For example, although few existing programs are 
that long, most practitioners agree that it will require at least 
6,000 items to cover thoroughly a one-semester course on an 
academic subject such as algebra. If we add 1 hr of homework 
for each hour of classroom instruction, a conventional course 
in algebra requires about 150 student-hours. This gives a ratio 
of about 40 items per student-hour under conventional study 
methods. 

We have already observed that Spiegel customarily allotted 
40 classroom hours for the training of billing clerks. The pro- 
gramed sequence that was substituted for the conventional 
method was about 1,000 items long a ratio of 25 items to one 
student-hour of conventional training. This would indicate that 
the Spiegel program was a little " thinner" than would be ex- 
pected by projecting the estimate for an academic subject. It 
should be borne in mind, however, that many items in the 
package -billing program required time-consuming clerical 
operations leading to the desired responses. 

Several estimates, based on both academic and industrial 
experience, place the time required for writing the teaching- 
machine program at 1/2-1 hr per item. This includes the 
preparation of a first draft, the important trial runs, and final 
revision of the program. A 1,000 -item industrial teaching- 
machine program would require, therefore, about 750 profes- 
sional man-hours of program time. Assuming that the direct 
cost of professional time is $10/hr and that the overhead is 
100%, the minimum cost of the program will be about $15,000. 
To this must be added the cost of the training materials or 
apparatus a factor that can vary widely with the degree of 
manual training required. 



A Case Study in Programed Instruction 



By KATHARINE H. HAIN and EDWARD J. HOLDER 
Schering Corporation 



Sobering Corporation has a staff of professional-service repre- 
sentatives that must be trained in the clinical and pharmacologi- 
cal background of the drug products principally new drugs that 
they present to physicians. In today's environment of rapidly 
widening medical knowledge and therapeutic discoveries, a con- 
stant search is necessary to discover new and more efficient 
training techniques with which to prepare these men to perform 
their function. To this end, a case study in programed instruc- 
tion was inaugurated. 

The purpose of this study was to compare the effectiveness of 
a new teaching technique, programed instruction, with that of 
conventional instruction in the training of Schering representa- 
tives. Although much additional experimentation and testing will 
have to be done before a definitive opinion can be reached as to 
the real value and place of programed instruction in our teaching 
curriculum, case reports of initial experiences can be helpful in 
providing information on what has been done so far, and they 
might suggest directions for future studies. This case study rep- 
resents our first step in experimenting with this new technique 
and, therefore, draws no conclusions but rather presents our 
tentative impression. 

THE INSTRUCTIONAL PROGRAM 

The program was designed to cover the clinical and pharma- 
cological background of griseofulvin, an antifungal agent for 
chronic fungous infections of the skin marketed by Schering Cor- 
poration under the trade name of Fulvicin. The programed ma- 
terial was, for all practical purposes, identical with the content 
of the traditional instruction given previously. The physician who 
administered the traditional instruction supplied the material in- 
cluded in the program. 

We decided to prepare the program for presentation by a pro- 
gramed textbook rather than by a teaching machine. The basis 
for this decision was threefold: machine presentation of programs 
has not yet been shown to possess any advantages over textbook 
presentation for this type of material, the programed-textbook 
format is considerably less expensive to prepare and administer 
than the machine format, and a programed textbook is more con- 
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venient to use than a machine (a textbook can be mailed and car- 
ried around with ease). 

An across-the-pages format was used in the programed text- 
book. The first frame appears at the top of the first page. After 
the student has written his answer to that question, he looks up 
the correct answer at the top of the following page, in the upper 
left. Still at the top of the second page, but now at the upper right, 
he finds the second frame. As soon as he has written his answer 
to the second frame, he turns to page three and again finds the 
correct answer in the upper left. 

The student proceeds in this manner until he reaches the end 
of the book; then he turns back to the beginning and again goes 
through the book, this time on another series of frames below the 
first. This across-the-pages format was considered superior to 
the down-the-page format because the latter results in undesired 
cueing through the appearance of consecutive frames on the same 
page, even though the large amount of page turning entailed by the 
across-the-pages format is admittedly undesirable. 

The program was of the constructed-response rather than the 
multiple -choice variety. The basis for this decision was twofold: 
one of the fundamental principles of learning theory states that 
the student must do that which he is to learn the student must 
learn to give the right answers, not merely to recognize them, 
and multiple -choice items require the student to study incorrect 
alternatives. (Of course, by studying them, the student learns and 
retains them to some extent, but he also learns errors that inter- 
fere with his later recall of the correct answers.) 

PREPARATION OF THE PROGRAM 

The instructional program was developed by Basic Systems, 
Inc., with the assistance of a medical team consisting of physi- 
cians at Grace -New Haven Hospital and the Yale University 
School of Medicine. Members of the Schering Corporation medi- 
cal and sales training staffs maintained liaison with the program- 
ing staff while the program was being prepared. The final pro- 
gram was 783 frames long. 

ADMINISTRATION OF THE PROGRAM 

To be able to compare the effects of the programed instruction 
and the conventional instruction, two comparable groups of new 
representatives were formed. 

Group 1 (the control group taught by conventional instruction) 
consisted of 19 men, 14 of whom had a Bachelors Degree, two of 
whom had a Masters Degree, and three of whom had no degree 
but some college training. 
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Before they convened at the company for a three -week training 
session, the men were informed that they would be trained and 
tested on griseofulvin. At the same time, a background brochure, 
25 pages long, containing the relevant scientific information was 
given to them. In addition, sample promotional material and bro- 
chures describing the features of the product were made available 
to them for study. The men were instructed to review the mate- 
rial prior to their arrival at the training session. 

During the training session, 4 hr and 45 min of instruction on 
griseofulvin that involved a combination of lectures and audio- 
visual presentations was administered by a physician member of 
the Schering Corporation clinical research staff. After this in- 
struction was completed, the group was told to study their notes 
and other available materials in preparation for an intensive ex- 
amination to be given two days later. 

Group 2 (the test group taught by programed instruction) con- 
sisted of 14 men, nine of whom had a Bachelors Degree, two of 
whom were Cuban M.D.'s with fair English skills, and three of 
whom had no degree but some college training. The programed 
textbooks were received by these men in the mail at home eight 
to ten days prior to their arrival at the training session. The 
men were simply told to complete the program and that they would 
have a test on it early in the training course. The group arrived 
at the training session on a Sunday night, and they were given the 
examination on Tuesday morning. 

THE RESULTS 

The average time for completing the programed text was 10 hr. 
No classroom instruction on griseofulvin was given Group 2 prior 
to the examination. The amount of time the men from the group 
receiving conventional instruction spent preparing for the exami- 
nation was not determined, but this group spent 4 hr 45 min in 
class. 

The examinations given the control and the test groups were 
similar but not identical because the two groups were not tested 
at the same time. The control group was examined in November 
and the test group in May. The examinations were purposely 
made harder and more comprehensive than those ordinarily given 
to a class of new representatives so as to thoroughly challenge 
the programed-instruction method. The examinations asked for 
a knowledge of a great deal more factual material than is required 
of new men who have just started training and who will continue 
to be trained in the field for many weeks following their classroom 
instruction at the home office. 

The mean grade for the control group who received the con- 
ventional instruction was 60.1. The mean grade for the test group 
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who received programed instruction was 91.9. The difference 
between the two means is significant by the t-test beyond the 0.01 
confidence level. 

Discussions were held with all members of the group taught 
by programed instruction. Their reaction to programed instruc- 
tion was unanimously positive. In response to a course -evaluation 
questionnaire, all members of the class said that they learned 
more with programed instruction than with the traditional teach- 
ing methods used in teaching them about other products. One man 
said that the program was too detailed, and one man said that it 
was not detailed enough. Two men considered it less interesting 
than the lectures and discussions on other products. 

On the basis of these results, our initial impression is that 
programed instruction gives promise of providing a new and use- 
ful teaching technique that could increase the learning efficiency 
of our professional service representatives and thus improve 
their training. This technique also gives promise of permitting 
reduction in our classroom teaching time and allowing more effi- 
cient and economical utilization of our professional training per- 
sonnel. 



The Mathetical Design of Effective Lessons 



By D. F. PENNINGTON and C. W. SLACK 
TOR Education, Incorporated 



The purpose of this article is to introduce the reader to the 
mathetical design of lessons and to mathetics as a method of les- 
son construction, Mathetics has a number of advantages over 
other methods linear and branch programing because it is more 
systematic and less restricted to special formats. 

As a technology, mathetics derives in detail from basic re- 
search in reinforcement theory with lower animals. In its appli- 
cation, it is assumed that man is at least an animal, and this as- 
sumption is carried through to the level of production and design 
details. One conclusion of mathetics is that the best and least 
expensive way to teach the human animal is to follow closely all 
that we now know about instilling and sustaining behavior in the 
lower animals. Through the recognition that he is at least an ani- 
mal, man is better enabled to distinguish within him that which 
is truly human and to direct his behavior toward human goals. 

One of the very best (and most difficult) initial tests of any 
technology is in industrial situations because real cost factors 
directly determine usage and eventual acceptance. Because ma- 
thetics uses inexpensive task- simulators rather than teaching ma- 
chines, does not make use of subject-matter experts to write 
frames, and does not test lessons until after completion, the cost 
of preparing lessons is lowered. 

CHARACTERISTICS OF THE METHOD 

A distinguishing characteristic of the mathetical approach lies 
in the method for producing the lessons. The result of an original 
approach based on reinforcement theory rather than a combina- 
tion of existing techniques, mathetics satisfied the following two 
criteria characteristic of a true technology: 

a. Reliability. Two mathetical teams working independently on 
the same educational objective (prescription) will produce simi- 
lar lesson materials, and two matheticists analysing the same 
task will generate similar prescriptions. 

b. Validity. The behavior contained in the prescription can be 
instilled into all members of the designated population; the pre- 
scription depicts the job or educational requirements accurately, 
provided that they can be behaviorally defined. 
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As stated previously, an advantage of mathetics is that task 
simulators are used rather than standard teaching machines. 
This permits mathetics to teach many kinds of behavior rather 
than just verbal and written behavior. Mathetics uses a trans - 
actional approach in which the trainee actually performs all the 
operations that will be required of him he learns by doing. The 
behavior to be instilled into the trainee can be derived from exist- 
ing training materials or from a subject-matter expert where 
good materials are not available. In any case, the behavior to be 
instilled into the trainee is prescribed in detail in advance and 
then systematically established in the repertoire in a manner that 
depends on the type of behavior involved. 

Sequences ("chains") in which each response sets up a unique 
stimulus "state of affairs" are taught backwards. Thus, a sales- 
man would be taught the "close" of a sale first, a machinist the 
finishing procedure first, and a bank-teller trainee the close and 
balance of his cage first. In this manner, the lesson establishes 
the goal firmly in the repertoire of the student or trainee. This 
procedure corresponds to the food pan in animal work. Once the 
human knows "where he is going", each step in the chain becomes 
a reinforcing stimulus by connection with the final state of affairs. 

Notice in the example of Fig. 7-3-1 that the first thing we teach 
is the last thing that a person reading a resistor does, namely, 
the reading of the tolerance band and the "plus -or -minus" pro- 
cedure. Lessons produced this way can greatly reduce the bulk 
of material required. 

In addition, with each exercise the trainee does more and more 
on his own until, at the end, he is performing the whole operation 
on his own. Also, at each point in the sequence he sees concrete 
evidence of his increasing knowledge and skillhe knows he can 
do more now than last time on his own. The speed, accuracy, 
and high motivation resulting from this approach are unequaled 
by other methods. 

All discriminations ("multiples") are brought to strength si- 
multaneously. Thus, a code or a language that involves large 
multiple discriminations is taught all at once. It can be shown 
that any other way of teaching greatly increases the time and de- 
creases retention. If, for example, we want to teach an animal 
to distinguish between two lights, we present both to the animal 
at the same time* Thus, in teaching a salesman to meet customer 
objections, such objections and their appropriate responses would 
be taught at the same time and brought to strength by repetition 
and decreasing use of intermediaries all at the same time. For 
instance, we teach an entire color code in one "frame" (see Fig. 
7-3-2). To teach the code color by color in either linear or 
branching fashion, would require many more frames. 
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4. The FOURTH COLOR BAND gives the PERCENT OF ERROE toler- 
ance in the resistance rating. The FOURTH BAND can have one of 
three colors; each COLOR stands for a PERCENT, Learn them. 

Accuracy with 5% of ratingFOURTH BAND is GOLD 
Accuracy within 10% FOURTH BAND is SILVER (silver-dime) 
Accuracy within 20% -FOURTH BAND is BLACK or MISSING 

NO ANSWER REQUIRED 

5. What is the tolerance percentage of a resistor if it is: 

% Tolerance 

(a) 400 ohms and the 4th band is MISSING? _ 

(b) 300 ohms and the 4th band is SILVER? _ 

(c) 200 ohms and the 4th band is GOLD? _ 

(d) 100 ohms and the 4th band is BLACK? 



(a) 20% (b) 10% (c) 5% (d) 20% 

Fig. 7-3-1 Two exercises selected from a mathetical lesson that teaches 
all the behavior involved in reading and electrical resistor. The lesson 
is twenty exercises long 

Generalizations are taught with what looks like, superficially, 
old-fashioned mnemonics. There is a great difference however. 
In teaching a generalization, we use an already existing response 
in the repertoire of the animal that is, also, a stimulus for the 
new response that we wish to establish. We thus create a small 
chain the center of which, the mnemonic, will tend to drop out 
with use (the animal always tends to shorten chains over time). 
When the mnemonic drops out, we are left with the desired stimu- 
lus-response (S-R) relationship. In Fig. 7-3-2, we can see that 
each color-number generalization is taught in minimum time with 
its own mnemonic bridge. Here we are putting competing behav- 
ior to work for us by strengthening a particular competing re- 
sponse that is also a strong stimulus for the desired response. 

COMPARISON OF MATHETICS WITH OTHER METHODS 

An analysis of the mathetical approach (see Table 7-3-1) shows 
that, even though it includes those parts of other methods that 
are compatible with the laws of animal behavior, it must be 
counted as a separate procedure. A distinguishing characteristic 
of other methods is their treatment of format as a policy matter 
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7. Each of the FIRST THREE COLOR BANDS can have one of 10 colors. 
Read through this list twice. Learn the NUMBER for which each 
COLOR stands 

. a FIVE dollar bill is GREEN . ZERO: BLACK nothingness 
. ONE BROWN penny . a RED heart has TWO parts 

. a WHITE cat has NINE lives . THREE ORANGES 
. SEVEN PURPLE seas . a FOUR legged YELLOW dog 

. a BLUE tail fly has SIX legs . an EIGHTY year old man has 

GRAY hair 

NO WRITTEN ANSWER REQUIRED 

8. List the number for which each COLOR stands: 

RED WHITE PURPLE BROWN BLACK 



(heart) (cat) (seas) (penny) (nothingness) 

GREEN _ GRAY _ BLUE _ ORANGE _ YELLOW _ 
(bill) (hair) (tail fly) (oranges) (dog) 



ANYTO 

Red 2_ White Purple 7. Brown _! Black 0. 

Green 5. Gray 8_ Blue 6_ Orange 3_ Yellow 4_ 



BLACK 


THROWN 


YELLOW 


rra AY 


ORE EN 


WHITE 


PURPLE 


RED 


oft AN OF: 


RTJTE 












Black 
White 9 


Brown 1 
Purple 7 


Yellow 4 
Red 2 


Gray 8 
Orange 3 


Green 5 
Blue 6 



Fig. 7-3-2 Three exercises selected from the same mathetical lesson as 
that of Fig. 7-2-1 

to be set a priori. This format is assumed to include the follow- 
ing: 

a. The ways of presenting the material once it has been pro- 
gramed. 

b. The kinds of responses (multiple -choice or constructed) 
that are allowed the student. 

c. The consequences of those responses to which the pro- 
gramer is limited. 

Partisans of the linear and branching approach differ on the 
best form for presenting a reinforcer and the kind of schedule 
on which it should be delivered. Both assume that outside indicia 
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Table 7-3-1 Comparison of Linear, Branching 
and Mathetical Approaches to Lesson Preparation 



Elements of 
lesson 
preparation 




Approaches 




Linear 


Branching 


Mathetical 


Presentation 


Decided in ad- 


Decided in ad- 


Decided near end of 


format 


vance 


vance 


lesson preparation 


Machine pre- 


Frame dis- 


Frame dis- 


Task simulators, us- 


sentation 


play with or 


play with 


ually of cardboard 




without sepa- 


multiple - 


or plastic 




rate answer 


choice 






tape 


buttons 




Book presen- 


Linear pro- 


' 'Scrambled' 7 


Varies with type of 


tation 


gram 


program 


behavior involved 


Frame 


Fixed frame 


Fixed frame 


Varies 




smaller than 


of full -page 






one page 


size 




Amount of in- 


Small 


Large 


As large as possible 


formation 








given in each 








step 








Number of 


Large 


Smaller 


As small as possible 


steps in les- 








son 








Errors al- 


In theory, 


One less than 


Some errors built in 


lowed by 


none; in 


number of 


to focus attention 


student or 


practice, 5% 


choices 


(some test items for 


trainee 


(some indi- 


given 


individualization) 




vidualiza- 








tion added) 






Try-out 


Every few 


Varies 


Only after entire les- 


testing 


frames 




son completed 




(written arid 








then tried 








out) 






Content 


Determined 


Determined 


Determined by stimu- 




by pro- 


by pro- 


lus-response analy- 




gram er 


gram er 


sis of subject 



Unit of 
measure 

Response 



The frame 



Constructed 
or written 



The frame 



Multiple 
choice 



matter 

The operant (amount 
of behavior that is 
changed by lesson) 

Combined task simu- 
lation and written or 
other construction 
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Elements of 
lesson 
preparation 




Approaches 




Linear 


Branching 


Mathetical 


Programers 


Should be sub- 


Should be sub- 


Need not be subject- 




ject-matter 


ject-matter 


matter expert 




expert 


expert 




Reinforcers 


Being right 


Knowledge of 


In general: task 




(being al- 


results 


completion and rate 




lowed to go 


(being right 


of increase. In par- 




on to next 


or wrong on 


ticular: each 




frame on 


variable 


stimulus in pre- 




fixed-ratio 


ratio) 


scription becomes 




of one) 




reinforcing stimu- 



lus connected to 
final " state of 
affairs" through 
chain. Student is 
almost always cor- 
rect 

Theoretical Immediate Socratic Immediate positive 

basis positive re- method reinforcement plus 

inforcement all other aspects of 

reinforcement theory: 
theory: chaining, 
discriminations, 
generalizations, 
scheduling, etc. 



of whether the student is "right" constitute the best reinforcer. 
Linear programers try to put the student on a fixed ratio of being 
right on each frame, whereas the branching programers use a 
variable ratio that allows the student to be wrong a certain por- 
tion of the time. 

The mathetical approach does not assume that outside indicia 
are the best reinforcers even when delivered immediately by a 
machine. Actually, students will work to be right (or to find out 
whether or not they are right) only to the extent that they have 
been deprived, in the past, of such information. After a while 
"immediate knowledge of results" looses its reinforcing strength 
because this strength derives only from our present educational 
methods, which deprive the student of success in this form. 

Initially, because both linear and branch programs look like 
tests, the student is gratified to get the answers right, but he may 
fail to complete the program if left on his own. From a strictly 
behavioral point of view, these programs effectively maintain the 
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behavior of reading and writing as long as the state of deprivation 
of success lasts; the teaching is incidental, and the student is left 
to devise his own method for learning the material. 

In the application of mathetics, each stimulus "state of affairs 7 ' 
in the final behavior is developed into a reinforcing stimulus 
through association with the final state of affairs. Outside infor- 
mation as to whether or not the response is correct is given only 
in the beginning to reassure the student or trainee and to give 
him confidence in his own correct behavior. 

MATHETICAL LESSON PREPARATION 

Mathetics makes use of an extensive function analysis and pre- 
scription in which the task or subject matter to be instilled is pre- 
scribed in terms of animal behavior (stimulus -response units 
called ope rants). 

Phase 1. Function Analysis 

9 

First, the educational and training objectives are defined in 
rather general terms according to an analysis of the real cost 
factors and other educational objectives. A reduction in training 
time or saving in personnel is not the only, or even the primary, 
objective. All self -instructional lessons and programs reduce 
training time and ease the load on personnel, but these, by them- 
selves, are seldom the major cost factors, even in industry. To 
give an example one of the major cost factors in the training of 
department-store employees is not training time, or even the 
load on personnel, but training space: classroom space and other 
space devoted to training. This factor is second only to the cost 
of operational reliability, that is, to mistakes made by trainees 
although, in the department-store business, it is not always the 
new employee who makes the most mistakes. 

In many other industries, ''drop outs" in training are a major 
cost factor. In some industrial situations, the real cost factor is 
the effective flow of trainees into operations so that the time of 
trained supervisory personnel is used to its greatest efficiency. 
In each situation a well-designed mathetical lesson would be de- 
signed and installed to reduce such real cost factors, which vary 
between and within industries and situations. 

A subject-matter expert is consulted prior to, or in conjunction 
with, the collection of data to define more specifically the task or 
educational content and the student or trainee. The subject matter 
expert does not advise the matheticist as the method of teaching- 
he advises only as to the elements essential to a mastery of the 
task being taught. 

Data collection involves more than just a detailed description 
of the elements traditionally related to a task. The actual ele- 
ments essential to a mastery of the task or content are sought, 
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and the usefulness of an element in the performance of a function, 
as opposed to the learning of that function, is determined. The in- 
clusion of elements for purely traditional reasons is discovered 
and the economy of an element in terms of frequency, importance, 
etc., is evaluated. 

Phase 2: Prescription of Behavior 

The "prescription" is a convenient and standardized way of 
representing the behavior to be taught. The basic unit of the pre- 
scription is the "operant," which consists of a single element of 
the behavior (R) plus its stimulus condition (S). Hence the operant 
takes the form of a stimulus -response relationship (S-R) 9 a series 
of which can exist in virtually any combination. Actually, any les- 
son is represented by both a "synthetic" (practice) and an 
"analytic" (theory) prescription, each of which serves a distinc- 
tive and essential purpose. 

The Synthetic Prescription. The final synthetic prescription 
is most commonly derived by a series of approximations directed 
toward a synthesis of the behavior expressed in optimally-sized 
units or operants. The first prescription is usually the most de- 
tailed because any question as to operant size is resolved by se- 
lecting the smaller, that is, if there is indecision as to whether 
a bit of behavior should be represented by one unit or two, the 
latter is chosen. This assures that all steps integral to the final 
operation are considered. 

Basically, there are three kinds of operant arrangements: se- 
quences ("chains"), discriminations ("multiples"), and generali- 
zations. A "chain" of operants is derived from behavior consist- 
ing of sequentially-ordered units, as in 



5 - ~R - S 



Skinner Pull Light Press Food Eat 

Box Chain on Lever Appears Food 

A "multiple" represents a series of contingencies with no essen- 
tial order, as in 



R 



red light turn right 



blue light turn left 



R 



white light remain 

A "generalization" represents several conditions requiring the 
same response, as in 
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name of state 



alphabetize 
first 



name of town 
Usually, some combination of the three is obtained, as in 



customer "cash 
selections or 
complete charge" 


5 **~R 
"cash" accept money 
and n6te 

"charge" "30-day" 
or "flexible 
charge?" 


o - - ~ 
currency 




place in cash register 


check 

o 


witness signature 
.y-R 


certificate 


note amount 



The second approximation to the final prescription derives 
from a refinement of the first, in which operants are collapsed 
so that any error is on the side of operants that are too large. 
(Note that any error away from optimally-sized units can be 
spotted easily in empirical test if an operant is too large but not 
if it is too small.) The final prescription contains all elements 
of the original but in a relatively condensed version. Thus, the 
synthetic prescription can appear as a complex of operations, 
each composed of a series of operants and combining to yield a 
rather mechanical picture of the total behavior to be taught. The 
second type of prescription is derived from an analysis of the 
operations so depicted. 

The Analytic Prescription. The operations of the final syn- 
thetic prescription are examined for common properties and col- 
lapsed into a complex of schematics. On the basis of the schema, 
a single "Archetype" problem (real or unreal) is sought that will 
serve as a basic model for all cases; i.e., one that involves all 
distinguishable operations. If such a problem cannot be found, 
the best compromise is used one that invites greater generali- 
zation but may be expressed in simple terms (a "paradigm" 
problem). The paradigm problem (or problems) is then prescribed 
and subjected to the same subsequent treatment as the synthetic 
material. 

Phase 3: Characterization of Behavior and Lesson Plan 

Characterization involves extensive consideration of each op- 
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erant comprising the final synthetic prescription, with particular 
regard to the following two sources of difficulty in teaching: 

1. Discrimination Difficulties. It is recognized that a basic 
quest in learning concerns the stimulus situation appropriate to 

a response (as opposed to the response itself), that is, it is easily 
demonstrable that virtually (perhaps literally) any segment of be- 
havior necessary to mastery of a practical task is already in the 
repertory of the learner. The crucial task is that of relating the 
behavior to its appropriate stimulus context. (A little reflection 
on the problems of learning to speak a foreign language is illus- 
trative.) Thus, of primary concern are the discrimination diffi- 
culties already learned or, more succinctly, the "incorrect" 
stimuli most likely to elicit a particular response. A recognition 
of such competing tendencies are systematically noted for later 
consideration and special treatment. 

2. Generalization Difficulties. Obviously, any stimulus situa- 
tion, when verbally described, can be only a symbolic approxi- 
mation to reality. We are here interested in the degree to which 
a given stimulus, taught in symbolic fashion, will generalize to 
all situations in which the response is approximate. Again, each 
operant is examined for probability of generalization failure and 
appropriate notation is made for later treatment. 

The lesson plan incorporates all elements heretofore developed 
and outlines the actual teaching sequence. The first exercise 
strengthens the last operant (for reasons that encompass a large 
bulk of reinforcement theory and will not be discussed here). 
This operant and each successive (or, more literally, succes- 
sively preceding) one is taught in three exercises, the first of 
which is described as a "demonstration," the second as a 
"prompt," and the third as a "release." Thus, a simple, three- 
unit chain and its lesson plan appears as 



Exercise Demonstrate Prompt Release 

1 ^? 7? 

O Q X? ^7? 

3C r> c r> C p 

OQ /t^ Oj^ -TL 2 *~>2 ^S 

4 $Q R l $1 R 2 

5 S RI 
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It can be observed that Exercise 1 serves to demonstrate the last 
operant; Exercise 2, to demonstrate the immediately prior op- 
erant and prompt the last one; etc. Parenthetically, there are 
always two more exercises than operants. 

Phase 4; Exercise Writing 

The lesson plan indicates that each operant is strengthened in 
three exercises but in a differential manner. 

The Demonstration. An operant is initially introduced in a 
very precise way. In addition to the reinforcement afforded by 
successful task completion, it is recognized that proper stimulus 
control is an integral part of the teaching process. For this rea- 
son, the following several aspects of the stimulus are routinely 
treated: 

a. The S^ is the actual stimulus situation that is to produce the 
response of a given operant. Obviously, such a stimulus situation 
usually cannot be presented per se. Rather, some symbolic re- 
presentation (usually verbal) must be used. 

b. The S a , literally the "attention" stimulus, locates for the 
student the S d (i.e., points out the $ d in such a way that it (the S<0 
cannot be overlooked or confused), and theS a identifies the essen- 
tial characteristics of the S^ so that it can be generalized and de- 
limited appropriately. Both aspects of the S ff must be present in 
every "demonstration" of an operant. 

c. The S v normally takes the form of a verbal instruction 
phrased in such a way that the response is forced (the student 
cannot reasonably make an error) in the presence of the S a . 

The Prompt. Each operant is prompted in the exercise follow- 
ing the one in which it was demonstrated. This is accomplished 
by means of a "residual" S v (the S v ), which provides the essen- 
tial elements of the S V 3 again in the presence of the most distin- 
guishing characteristic(s) of the S^. 

The Release. In the exercise following its prompting, each 
operant is strengthened by a repetition elicited by the preceding 
response. (In a chain, each response produces the stimulus of 
the next operant.) It can be noted that, with the exceptions of the 
first two and the last two, each release strengthens three operants. 
Thus, in the sample lesson plan presented, Exercise 3 serves to 
demonstrate the operant S -R 19 to prompt S l -R 2 and to release 
S 2 - # 3 . (The release of the last operant in a chain can be elicited 
by the simple verbal instruction, "Now complete the operation," 
following the prompting of the preceding operant.) 

Phase 5: Task Simulation and Format Design 

At this point (and not until this point) consideration is given to 
the form of the final lessons. Normally, the exercises are readily 
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suggestive of a gross format (a programed book, teaching ma- 
chine, kit, etc.). Anyone who insists on a particular format in 
advance should be cautioned that such a format might not be the 
most efficient one for his particular task. Three primary con- 
siderations are paramount here: 

a. The probability of generalization to the actual performance 
situation, 

b. Comparative efficiences of fitting various formats to the 
exercises (relevant factors include length of exercises, kind of 
behavior (whether primarily verbal or motor), types of symboli- 
zation employed, nature of the operant components, space and 
facilities available to the students, etc.). 

c. Production (fabrication) feasibility and costs. 

Again, it must be emphasized that any precise restrictions im- 
posed by the client are often to his disadvantage because the most 
appropriate format is seldom if ever discernible prior to the 
completion of all preceding phases. 

Phase 6: Editing and Evaluation 

The writing of the exercises as described assures that all 
technical components are introduced. It does not emphasize effi- 
ciency of presentation of these components, which is one of the 
functions of editing and evaluation. Editing and evaluation is re- 
sponsible for three operations: rewrite and layout editing, copy- 
editing, and try-out editing. 

The rewrite and layout editor is concerned solely with the 
communic ability of the language and the related illustrations of 
a lesson. Though trained to recognize and to know the function 
of the components, the person responsible for rewriting and lay- 
out editing never concerns himself with their accuracy or their 
adequacy only with whether they are properly communicated to 
the trainee. 

The copy editor assures grammatical correctness, clarity, and 
consistency of language and format. He approaches the lesson 
from the trainee's point of view. He must return the lesson to 
the rewrite and layout editor for approval of any changes. The 
lesson then is reviewed by an editor to verify that the mathetical 
components have not been distorted or deleted in the preceding 
operations. 

Try-out editing involves testing a lesson with a selected "sub- 
ject trainee" who most nearly conforms to, or is actually of, the 
trainee population. It is within this step that all aspects of the 
lesson are evaluated especially the mathetical components. Spe- 
cial procedures are applied whereby each aspect of the lesson 
can be readily identified and appraised by the try-out editor 
throughout the testing session. 
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It should be noted that not only after the try-out operation, but 
at any point, a lesson can be returned for revision to an earlier 
stage of development if such is deemed necessary. The matheti- 
cist, to exercise maximum control, deliberately builds into his 
lessons possible sources of error; therefore, lessons are rede- 
signed, re -edited, and re -evaluated as many times as necessary 
to achieve the criterion product. 

Phase 7: Installation 

The installation procedures of a mathetical lesson, in large 
part, determine its success in business, industry, education, etc. 
Though different in its obvious form, a lesson, like any new 
"equipment/' must be installed properly, used properly, tested 
properly, and given proper servicing. Thus, the proper installa- 
tion steps must be followed closely to achieve the best results. 



For most of the ideas contained in this article, credit is given 
to Gilbert, T. F. (1962) Mathetics, the technology of education, 
Journal of Mathetics, Vol. 1, No. 1. The authors, however, as- 
sume full responsibility for the adequacy of expo si ti on < 
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APPENDIX A: THE FIRST 10 FRAMES (OUT OF 719) OF 
THE 7070 PROGRAM PREPARED BY IBM 

Frame 1 

The 7070 is a data processing sys- 
tem. To prepare a payroll, to main- 
tain an inventory, or to perform other 
accounting applications, a customer 
can use the 7070 data 



PROCESSING 
SYSTEM 



Frame 2 

All data processing systems require 
some type of input unit or units. In 
order to put information into the 7070 



an 



unit is required. 

Frame 3 

In addition to one or more input units 
all data processing systems require 
some sort of processing unit or units to 
operate on the input data. The 7070 has 

several 

units to process input data. 

Frame 4 

Data are put into the 7070 by means 

of an unit. The 

information (data) is then operated up- 
on by several 



DATA PROCESSING 
SYSTEM INPUT 



PROCESSING 



INPUT 

PROCESSING 
UNITS 



Frame 5 

All data processing systems must 
also have output units to write out in- 
formation in a usable form. A me- 
chanical printer is an example of an 

unit. 



OUTPUT 



313 



314 

Frame 6 

The 7070 data processing system 
can perform various accounting appli- 
cations through the use of its various 
basic units. These basic units consist 

of units 3 

_ units, and 
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units . 



Frame 7 
The 7070_ 



can have various types of input units. 
The 7500 Card Reader is one of the 

units for the 7070 system, 

To read card information into the 
7070 we use the 



Frame 8 

To read punched card information 

into the 7070, we use a 

Card Reader. 

Frame 9 

The 7500 

unit reads card at the rate of 500 cards 
per minute (C.P.M.). Since a maximum 
of three 7500 units can be used with the 

7070 system, a total of 

cards per minute (C.P.M.) can be read 
into the 7070. 

Frame 10 

A maximum of three Card 

Readers can be used with the 7070 sys- 
tem. Each Card Reader can read cards 

at the rate of C.P.M. 

It will help you to recall this by remem- 
bering that the Card Reader 

unit handles C.P.M. 



INPUT 

PROCESSING 
OUTPUT 



DATA 

PROCESSING SYSTEM 

INPUT 

7500 CARD 

READER 



7500 



CARD READER 
1500 



7500 
500 

7500 
500 



APPENDIX B: SAMPLE OF 10 ITEMS FROM AN 88-ITEM 
ACHIEVEMENT EXAMINATION PREPARED BY IBM 

5. Identify which of the following units use the magnetic drum 
for buffer storage. 

a. 7400 Printer 

b. 7501 Console Card Reader 

c. 7550 Card Punch 

d. 7500 Card Reader 

e. 7900 Inquiry Station 

12. What is the maximum number of typewriters in the 7070 
system? 



20. Draw a block diagram of the complete 7070 system. Label 
each block with name and number. Draw arrows to indicate the 
data flow paths. 



23. The following bit positions make up a word in storage. 
What is the word? 

03 16 01 16 06 16 23 06 23 26 02 



50. Give the field defined digits and their sign for the instruc- 
tion words below: 

Data Words: 

0800 = -8274316721 
1234 = +2364510793 

Instruction Words: Sign Digits 

+ 14 00 09 0800 

-14 01 00 1000 

+ 14 00 02 1243 

-23 01 36 1443 



Index Word: 

0001 = 0002000700 

52. What is the effective address portion for each of the follow- 
ing instructions: 

Given: 

IW 01 = -0074532640 
IW 02 = +0033642751 
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Instruction Words: Effective address portion 

A. +14 01 09 0700 

B. +13 02 27 0800 

C. -14 01 09 0200 

D. -50 02 69 8420 

E. -13 01 27 0642 



59. In a phrase or sentence, give the function of the Instruc- 
tion Cycle. 



69. What registers (10 digits and sign) are used when reading 
from a 7500 Card Reader? 



79. Draw a flow chart of the program step showing the various 
cycles. 



88. Draw a block diagram of the information, arithmetic, and 
distributor buses showing their associated registers capable of 
handling 10 digits and a sign. Indicate all connections. 



APPENDIX C: RESPONSES OF AN EXPERIMENTAL 
GROUP (n = 70) TO AN IBM STUDENT QUESTIONNAIRE 

Compared to the regular classroom method used in other IBM 
courses you have taken: 

1. How well has the programed instruction (PI) method taught 
you the material covered? 

D PI much less effective than regular classroom 
method 

Q PI somewhat less effective than regular classroom 
method 

D PI about the same as regular classroom method 

21% Q PI somewhat more effective than regular class- 
room method 

79% D PI much more effective than regular classroom 
method 

2. How do you like the programed instruction (PI) method? 

3% D Like PI much less than regular classroom method 

3% D Like PI somewhat less than regular classroom 
method 

7% D Like PI and regular classroom method about the 
same 

24% D Like PI somewhat more than regular classroom 
method 

63% D Like PI much more than regular classroom method 

3. How difficult was it to learn using the programed instruc- 
tion (PI) method? 

D PI much more difficult than regular classroom 
method 

D PI somewhat more difficult than regular classroom 
method 

7% D No difference in difficulty between regular class- 
room method and PI 

31% D PI somewhat less difficult than regular classroom 
method 
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62% D PI much less difficult than regular classroom 
method 

4. How much home study does the programed instruction (PI) 
method require? 

3% D PI requires much more home study than regular 
classroom method 

13% D PI requires somewhat more home study than regu- 
lar classroom method 

24% D PI and regular classroom method about the same 

36% D PI requires somewhat less home study than regular 
classroom method 

24% D PI requires much less home study than regular 
classroom method 

5. In future IBM courses you may take, would you like to see 
the programed instruction (PI) method used in place of the reg- 
ular classroom method? 

D Strongly object to using PI in place of regular class- 
room method 

13% D Would have some objections using PI in place of 
regular classroom method 

4% D Don't care which method is used 

37% D Have some preference for using PI in place of 
regular classroom method 

46% D Strongly prefer using PI in place of regular class- 
room method 

6. Was there anything about programed instruction that you 
particularly liked? 

Learning of material much easier. Information presented 
in a very clear manner. 

Student can proceed at his own learning rate. 

Instead of listening and note taking, the act of actually 
reading and answering keeps one's attention more alert 
and much less apt to miss anything. 

Step-by-step building up of principles with ideas repeatedly, 
constantly when important. 
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Covered much more material than the classroom lecture 
in the same time period could have covered effectively. 

7. Was there anything about programed instruction that you 
particularly disliked? 

Too much turning of paper. A machine would be nicer. 
The amount of writing and the repetition. 

Too much can become somewhat boring. I feel method 
should be used with moderation along with conventional 
classroom techniques. 

You should not be times. Time allotted for each unit 
seems to be geared for very rapid readers. 

Material as written is not too well adapted for review 
work since it requires so much time to re-read. 

Some sections were much too basic. The PI course 
should be geared to the individual student. This is one 
advantage of PI ... where in the lecture the instruction 
would have to start his lecture at the lowest point. 

8. Use the space below to make any other comments you wish 
to make regarding programed instruction. 

At the end I would like to see a summary in paragraph 
form or block form. 

Use of this should be continued, but not in entirety. It 
should be broken up with classroom instruction. 

Program should be expanded to cover more of course. 
On the whole, I believe this is the best method of learn- 
ing I have ever seen. 

This section should be sent into the field for C.E.'s to 
complete before they come to Poughkeepsie. 

A good method but time should be taken for some can- 
not read as fast as others. 
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APPENDIX E: THE FIRST 50 FRAMES OF A PROGRAM IN 
LIBRARY USAGE FOR JUNIOR HIGH SCHOOL 
Prepared by Automated Teaching Systems, Inc.* 



Books in a library are located in two separate 
sections. 

a) fiction 

b) non -fiction 

A romantic novel about the Civil War would be 
found in the section. 

a) fiction 



A history of the Civil War would be located in the 
(select "&" or "b") section. 

a) fiction 

b) non-fiction 

b) non-fiction 



In libraries, fiction books and non-fiction are lo- 
cated on (the same/different) shelves. 

Would a detective story and a geography text -book 
be on the same shelf? 

different 
no 



*M. Robb, S. Margulies, and I. A. Goldberg 
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In some libraries, fiction books are placed on the 
left side of the room and non-fiction books are placed on 
the right side. 

Which of the following types of books would be on 
the left side: 

a) a detective story 

b) a how -to -do -to book on stamp collecting 

c) an arithmetic text -book 

d) a book of poems 



a) a detective story 



Books in the library are separated into two sec- 
tions, and . 



fiction 
non -fiction 



There is an alphabetical list of all the books in the 
library. 

Would both fiction and non -fiction books be in this 
list? 



yes 



The alphabetical list of library books is printed on 
file cards. 

This list is called a c d catalog. 

card 
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Sears and Roebuck prints a book which lists seeds. 
This list is called a seed catalog. 

The alphabetical list of library books is called a 



card catalog 



The card catalog lists (some/all) of the books in 
the library. 

The easiest way to find out if your library has a 
copy of "The Three Musketeers 77 is to look in the 



all 

card catalog 



10 



Books in a library are separated into 


and 


> 




Every book in the library is listed in the 


. 







fiction 
non-fiction 
card catalog 



11 



People's names in the telephone book are listed in 
alphabetical order of last names. 

Which of the following names is listed first: 

a) Ben Jones 

b) Sam Davis 



b) Sam Davis 
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12 



As in the telephone book, author's names in the 
card catalog are listed in alphabetical order of last names. 

If you wanted to find the name of Mark Twain, would 
you look under Mark or Twain? 



Twain 



13 



Which of the following names would be listed first? 

a) William Shakespeare 

b) Samuel Clemens 

c) Nathaniel Hawthorne 



b) Samuel Clemens 

14 



The cards in the card catalog are like the names in 

the telephone book because they are both arranged in 

order. 



alphabetical 

15 



If you wanted to know whether your library has a 
book by Abraham Lincoln, you would look it up under his 
(first/last) name in the card catalog. 

last 
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16 



799.2 Boyle, John 
B46 Shooting for Deer in 
Northern Michigan 



This is a reduced copy of a card from the card 
catalog. (The actual card is approximately 3 inches by 5 
inches in size.) 

The author's name appears on the (top/bottom) of 
the card. 



top 



17 



Which of the following authors would be listed after 
the other two in the card catalog: 

a) Booth Tarkington 

b) Bret Harte 

c) Jack London 

a) Booth Tarkington 



18 



429.13 Haskins, Charles 
HI 4 How to Identify 
Airplanes 



This card comes from the card catalog. 

The author's name appears on the (top/bottom) of 
the card. 

Is the title of the book also on the card? 



top 
yes 
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19 



This Cc 
The au 
The bo 


F Treasure Island 
Robert Louis Stevenson 




ird also comes from the catalog, 
thor's name is on (top/bottom), 
ok title is on (top/bottom). 


bottom 
top 



20 



Some catalog cards have the author's name on top. 
Some have the book-title on top. 



All catalog cards have the_ 
the book ______ _. 



name and 



author's 
title 



21 



If you look at a card with the author's name on top, 
will it always have a title of the book underneath? 



yes 



22 



Cards with the author's name on top are called 
author cards. Cards with the book title on top are called 
title cards. 



F Tom Sawyer 

Clemens, Samuel 



Is this an author card or a title card? 



title card 
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23 



F Steinbeck, John 

The Grapes of Wrath 



This card is an 



author card 

24 

A catalog card has "Huckleberry Finn" printed at 
the top. 

This card is a card. 

Is the author's name also printed on the card? 

title 

yes 

25 

As with author's names, book titles are also listed 
in alphabetical order in the card catalogs. Titles are listed 
in order of the first word. 

Which of the following book titles would be listed 
first: 

a) Huckleberry Finn 

b) Captain Blood 

c) Two Years Before The Mast 

b) Captain Blood 

26 

"Black Beauty" would be placed (before/ after) "Two 
Years Before The Mast" in the card catalog. 

before 
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27 



All the cards in the card catalog are arranged in a 
single alphabetical order. The title cards are not located 
separately from the author cards. 

What would be the order of the following cards: 



a) 
b) 
c) 



629.13 How to Identify Airplanes 
414 Haskins, Charles 



F Call of the Wild 
London, Jack 



F Sabatini, Rafael 
Captain Blood 



b) 
a) 
c) 



28 



If the first word of the title of a book is The or A or 
An, the title card is listed in alphabetical order of the 
second word. 

The card for "The Swiss Family Robinson" would 
be filed under (select "a" or "b"). 

a) The 

b) Swiss 



b) Swiss 



29 



The card for "The Swiss Family Robinson" would 
come (before/after) the card for "Tom Sawyer''. 



before 



330 APPENDIX E 

30 

Which of the following author's names or book titles 
would appear first in the card catalog: 

a) The Iliad 

b) Jack London 

c) National Velvet 



a) The Iliad 



31 



The library contains both fiction and non -fiction 
books. Fiction books have the letter F at the left of the 
catalog card. Non-fiction books have a number at the left 
of the catalog card. 



F Steinbeck, John 

The Grapes of Wrath 



"The Grapes of Wrath" is a_ 



book. 



fiction 



32 



"Ho 




book. 


629.13 Haskins, Charles 
H. 14 How to Identify 


Airplanes 


w to Identify Airplanes" is a 



non -fiction 



33 



To find out whether a book is fiction or non-fiction, 
you would look in the . 



card catalog 



APPENDIX E 331 

34 



F Shakespeare, William 

A Midsummer Night's Dream 



What is the title of this book? 
What is the author's name? 
Is it fiction or non-fiction? 



A Midsummer Night's Dream 
William Shakespeare 
fiction 



35 



A list of all the books in the library is called the 



Each card gives both the and the 

of the book. 

The card also tells whether the book is fiction or 



card catalogue 

author's name 

title 

non -fiction 



36 



If the first word of the title of a book is an article 
such as A, An or The, it is omitted from the title on top of 
the catalog card. 

This is done because (use your own words). 



these words are not important to the titles, or there 
are so many titles starting with these words. 
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37 

The books on the fiction shelves of the library are 
arranged in the alphabetical order of author's last names. 

Would "Oliver Twist" by Charles Dickens and "The 
Three Musketeers" by Alexander Dumas be near each 
other? 



yes 



38 



"Oliver Twist" would be placed on the fiction shelf 
(before/ after) a book by Thomas Abbott. 

It would be (before/ after) a book by George Stine. 



after 
before 



39 



Could you easily find a book on the fiction shelves 
of the library if you only knew the title and did not look up 
the author's name in the card catalog? 



no 



40 



If you wanted to find a book on the fiction shelves 
and you only knew the title, you would look up the book title 
in the card catalog. 

This card would tell you the . 



author's name 

41 

If you wanted to find a fiction book called "The 
Prince and The Pauper" and did not know the author, what 
word would you look for in the card catalog? 

Prince 
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42 



In most libraries special types of fiction books such 
as Mysteries and Westerns are kept on separate shelves 
from the rest of the novels. 

Are these books listed in the card catalog? 



yes 

43 



The shelves for special types of fiction are identi- 
fied by signs which show the type of book. 

If you wanted to read a Mystery, but did not have any 
particular one in mind, you would go to the special shelf 
marked 



MYSTERY 

44 



The cards in the card catalog have a special symbol 
for books on special shelves. For example SS on a card 
means that the book is on the Short Story shelf. 

If the catalog card had the symbol PR on it, which 
of the following shelves would include the book: 

a) MYSTERY 

b) DRAMA 

c) WESTERN 



b) DRAMA 

45 



If a catalog card had the symbol for Mystery on it, 
would the book be in the non-fiction section of the library? 

Where would it be? 



no 

on the special fiction shelf for Mysteries 
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46 



F Murder in the Garden 
Sayers, Dorothy 



Would this book be on the special shelf for Mys- 
teries? 

In which part of the library would you look for it? 

no (if you were wrong, please go back to step 42 and 
review) 

On the fiction shelves, among the_S_ authors 

47 

If you looked for a book in the card catalog and the 

530 
card had r ,_ 1 on it, the book would be in the section 

VjrO 1 

of the library. 

non -fiction 



48 

All books in the non-fiction section of the library 
have two numbers printed on the back of the book cover. 
These numbers are arranged one over the other. 

Qjn OKI 4.70 

' A18- or LI 2 are a ^ exam P* es f these numbers. 

The numbers for each book are also printed on the 
of that book. 

catalog cards 
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49 



All the non -fiction books are arranged in numerical 
order of the top numbers which are printed on the book 

covers. All fiction books are arranged in order 

of the author's name. 



alphabetical 
last 



50 



The books in the non -fiction section of the library 

are arranged in order of the printed 

on the book covers. 



top numbers 



APPENDIX F: THE FIRST 46 FRAMES OF A PROGRAM 
IN STOCKS, BONDS, AND THE OPERATION OF THE 
STOCK EXCHANGE 

Prepared by Automated Teaching Systems, Inc. * 



1 



A railroad needs $950,000 to buy tank cars. It 
issues 1,000 bonds which are offered for sale to the public 
at $950 each. 

If all the bonds are sold, the railroad will realize 

$ . 



$950,000 



At the end of a ten year period, each railroad bond 
which was sold will be redeemed by the railroad at its face 
value of $1,000. 

Each bond costs $950 at purchase. At the end of ten 

years, each bond will make a profit of $ for 

the bondholder. 



$50 



The day on which the face value of a bond is re- 
deemed is called the date of maturity. 

If 1,000 bonds are sold, each having a face value of 
$1,000, the total amount which must be returned to the 
bondholders on the date of maturity is $ . 

$1,000,000 



*J. Reid (edited by S. Margulies and I. A. Goldberg). 
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4 



The total profit to an investor who purchases a 
utilities -bond for $1,200 with a 20 -year maturity value of 
$1,500 is $ . 

The profit per year is $ . 

The total profit per year from two such bonds would 

be $ . 



$300 

$15 

$30 



The day on which a bond is redeemed at face value 
is called the . 



date of maturity. 



In addition to its growth from purchase price to face 
value at maturity, a bond also pays annual interest on its 
face value. 

If you purchase a $1 ? 000 -(face value) railroad-bond 
having an annual interest rate of 3 1/2%, the bond would 
earn $ per year. 

$35 



338 APPENDIX F 

7 

A utilities -bond has a face value of $10,000 at ma- 
turity after 25 years. It sells for $9,200 and pays an annu- 
al interest of 3%. 

The annual interest earned is $ . 



The profit from the growth of the bond to its matu- 
rity value is $ when it is redeemed. 

The accumulated total return to the investor from 

all sources over and above the purchase price is $ 

when the bond is redeemed. 

$300 
$800 

$8,300 (this appears to be a large return, but re- 
member that it takes 25 years to accumulate) 



Which of the following does not represent a source 
of return on a bond investment? 

a) Interest 

b) Dividend 

c) Growth to maturity 

b) Dividend 



By issuing bonds, a company has a means of borrow- 
ing money. 

To a bondholder, each bond represents (select "a" 
or "b") 



a) part ownership of the company. 

b) a debt of the company to him. 



b) a debt of the company to him. 
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10 



A security is a certificate which may represent 
either ownership or debt. 

One type of security representing debt is called 



bond 



11 



The obligation to pay the interest and the matu- 
rity value of a railroad-bond must be met by the (select 
"a", "b ?? or "c* ? ) 

a) New York Stock Exchange. 

b) issuing company. 

c) U. S. Government. 



b) issuing company. 

12 



The two obligations of a bond issuing company to 
its bondholders are and . 



payment of interest on face value (any equivalent 
statement is correct) 

redemption at maturity of face value (any equivalent 
statement is correct) 



13 



The money that Firestone makes from selling tires 
is called earnings, Firestone assigns part of its earnings 
to pay obligations on bonds. 

If 1961 earnings are not sufficient to meet its bond 
obligations, Firestone (can/cannot) legally skip its bond- 
interest payment? 



can (the payment must be made up in a future year) 
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14 

The money that Ford makes from selling cars is 
called . 



earnings 

15 

In addition to bonds, a company may sell a type of 
security called common stock to raise funds. 

If a railroad issues 500,000 shares of common stock 
which are brought by the public at a price of $25 per share, 
the railroad will realize $ . (Ignore selling fees) 

100 shares will cost $ . (Ignore purchase 

fees) 

$12,500,000 
$2,500 



16 

The price of common stock may fluctuate with public 
demand. 

XYZ Corp. has very good prospects and many people 
want to buy its common stock. The per -share selling price 
goes up from $15 to $17. 

If you had purchased 100 shares at the lower price, 
and sold them at the higher price, your profit would be 
$ . (Ignore brokerage fees) 

$200 
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17 



The XYZ Corp. manufactures railroad cars, but 
more people are traveling by plane and the future growth 
of railroads does not look favorable. 

In this case, what may happen to the per -share 
price of XYZ Corp. common stock? 



It will probably go down (any equivalent statement 
is correct) 



18 



Money made from the sale of stock at a profit is 
called capital gain. 

What is capital loss? 



Money lost from the sale of stock at a lower price 
than originally purchased. (Any equivalent statement is 
correct) 



19 



In addition to potential capital gains, a stockholder 
may also get a return on his investment by means of a 
common stock dividend. 

If a dividend of $1 per share is declared, the total 
return on 300 shares is $ . 



$300 

20 



A common stockholder may get a return on his in- 
vestment by,_____gain and payments. 



capital 
dividend 
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21 

Like bond interest payments, common stock divi- 
dends are paid out of company , 

earnings. 

22 

Stock dividends usually reflect earnings. 

When earnings are high, dividends are increased to 
give the stockholder a higher return on his investment. 

Bond interest payments (also vary/ do not vary) as 
company earnings go up or down? 

do not vary 



23 

Bond obligations are considered as fixed charges to 
be paid by the issuing company. 

Stock dividend payments (are also/ are not) con- 
sidered as fixed charges. 

are not 



24 

When company earnings are distributed, fixed bond 
obligations are paid off first. 

Remaining earnings may be paid as dividends to the 
shareholders of . 

common stocks (or stocks) 
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25 



If annual earnings are low and there is only suffi- 
cient money to pay fixed bond obligations, a dividend (will/ 
will probably not) be paid to common stock shareholders. 



will probably not 



26 



When company earnings are high, return on invest- 
ment is greater for holders of than for 



common stocks 
bonds 



27 



Some companies have managed to pay a regular divi- 
dend for the past 20 years. 

The market price of the common stock of these com- 
panies should be relatively (select "a" or "b") 

a) high 

b) low 



a) high 

28 



Which of the following is more likely to be skipped 
when earnings are low? 

a) Dividends 

b) Bond interest payments 



a) Dividends 
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29 

Match the security with its basic advantage to the 
holder. 

a) Common stock A) Offers smallest risk of losing money 

b) Bond B) Offers opportunity for higher return 

a) Common stock- -B) Offers opportunity for higher return 

b) Bond -~A) Offers smallest risk of losing money 



30 

Company A, a toy company with few physical facili- 
ties, has high earnings due to a recent success of one of 
its products. 

Company B, a steel company with vast physical 
holdings, is presently operating at a loss. 

Company offers the better common stock 

investment. 
Company is the better bond investment. 

A 
B 



31 

Is it possible for a bond investment in one company 
to be more risky than a common stock investment in an- 
other company? 

Yes (a bond held with a company which may go 
bankrupt and not have sufficient assets to 
cover its bond obligations is clearly a more 
risky investment than common shares in a 
powerful, deeply rooted company) 
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32 



Ultimately, the soundness of any security depends 
on the soundness of the 



company 

33 



A share of stock represents part ownership in a 
company. 

Ownership (is also/is not) represented by a bond. 



is not 



34 



A bond represents a of the issuing company. 

Both stock and bond certificates are called 



debt 
securities 



35 



A common stock shareholder has a voice in the 
policies of a company because stock represents 



ownership 
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36 



One share of common stock carries the right of one 
vote at stockholders meetings where company policy is 
decided. 

Which of the following is sufficient for control of 
company policy? 

a) Ownership of 50% of common shares 

b) Ownership of at least 51% of common shares 

c) Ownership of 95% of company bonds 

d) Ownership of 35% of company shares 



b) Ownership of at least 51% of common shares 

37 



The two types of securities which have been discus- 
sed so far are__ and 



bonds 
common stocks 



38 



A third type of security is called preferred stock. 

Like the owner of common stock, the investor in pre- 
ferred stock can get a return from 



capital gain 
dividends 
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39 



Preferred stock has a claim on earnings which is 
prior to that of common stock but after that of bonds. 

List the order in which the securities have claim 
to earnings: 

1st priority -- 

2nd priority -- 

3rd priority -- 



1st priority --Bonds 

2nd priority -- Preferred stocks 

3rd priority -- Common stocks 



40 



Many preferred stocks pay a fixed dividend in regu- 
lar quarterly, semi-annual or annual payments. 

This type of preferred stock dividend payment is 
similar to payment of (select "a" or "b"). 

a) bond interest 

b) common stock dividend 



a) bond interest 



Some preferred stocks have a cumulative dividend 
feature. 

With these stocks, if the issuing company skips one 
or more fixed dividend payments, it must pay off all divi- 
dend arrears before a common stock dividend can be de- 
clared. 

Does the cumulative dividend feature guarantee the 
preferred stockholder of getting his fixed dividend? 



No (the company may never have sufficient earnings 
to pay fixed dividends) 
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42 



A man owns 400 shares of preferred stock paying a 
fixed annual cumulative dividend of $3. The company misses 
5 dividend payments. At the end of the sixth year, it de- 
clares a common stock dividend 

The total dividend which must first be paid to the 
holder of the 400 preferred shares is $. . 



$7,200 (6 x $1,200) 

43 



Under some conditions, it is possible for a company 
to pay a higher dividend on common shares than the fixed 
dividend paid on preferred shares. 

This is most likely to occur when earnings are 
(high/ low). 

In this case, it would be more profitable to hold 
shares than shares. 

high 

common 

preferred 



44 

There are many features which are designed to 
make preferred stocks more attractive than common stocks, 

One such feature is the payment of , 



cumulative dividends (or fixed dividends) 
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45 



Preferred stocks are an attempt to combine the best 
features of bonds and common stocks. 

They provide a higher return than bonds and are 
considered to be a invest- 
ment than . 



safer (any equivalent statement is correct) 
common stock 



46 



A Stock Exchange provides a place where securities 
may be traded between buyers and sellers. The Exchange 
itself does not issue securities, nor does it buy or sell 
them. 

Is it possible to buy 10 shares of IBM common stock 
from the New York Stock Exchange? 

No 



APPENDIX G: A FOUR-PAGE SAMPLE SEQUENCE IN 

ARITHMETIC 

Prepared by Norman A. Crowder 



page 1 



We have learned that the numbers multiplied together to 
form a product are called the factors of that product. 

There are some interesting and important results when the 
same number is used as a factor several times. A knowledge of 
this process is essential to an understanding of exponents, loga- 
rithms, the slide rule, the operation of certain computers, and 
of many other powerful mathematical tools. 

Here's an example of the same number used as a factor 
more than once: 

In the multiplication 
3x3 = 9 

the number 3 appears as a factor twice. Of course we can use 
the same factor more than twice. 

What is the product when 2 is used as a factor 3 times? 
Turn to the page which corresponds to the right answer. 

6 page 3 

8 page 6 

9 page 7 
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page 3 
(from page 1) 

YOUR ANSWER: When 2 is used as a factor three times, the 
product is 6. 

You merely used 2 and 3 as factors, 2x3 = 6. This is 
incorrect. 

We want the product you would get if you used the number 2 
as a factor three times. In other words, we want the result of 
the multiplication 

2x2x2=? 

Now return to page 1 and choose another answer. 
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page 6 
(from page 1) 

YOUR ANSWER WAS U 8". 
You are correct. 
2x2x2=8 

The mathematical symbol meaning "form the product 
reached by using 2 as a factor three times" is 

2 3 
It means 

3 ~ this * n ? times 



use the number 2 3 
as a factor. 

Similarly, 2x2x2x2 could be written as 2 4 , 3x3 as 3 2 , 
etc. 

What would 3 4 be? 

3 4 = 3 x 4 = 12 page 2 

3 4 =4x4x4= 64 page 5 

3 4 =3x3x3x3 = 81 page 8 
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page 7 
(from page 1) 

YOUR ANSWER: When 2 is used as a factor three times, the 
product is 9. 

Your answer is incorrect. You have used the number 3 as a 
factor twice. 

3x3 = 9. 

However, you were asked to use the number 2 as a factor 
three times. 

2x2x2=? 
Now return to page 1 and choose another answer. 
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Adrenal glands 
Thyroid gland 



Pituitary gland 
Parathyroid gland 



Ail of the above organs secrete substances which affect processes 
in various parts of the body. What word do the names of these 
organs have in common? 



: TURN PAGE FOR ANSWER AND QUESTION 2. ) 



21. 



( THIS IS NOT QUESTION 2. QUESTION 2 IS ON THE NEXT PAGE. ) 



An impoitant class of hormones known as the steroids is produced 
only by the testes, the ovaries, the placenta and the adrenal 
cortices (plural of cortex). The steroids produced by the 
adrenal (plural) are called corticosteroids. 



41. 



A stalk joins the pituitary gland to the underside of 
the 



When the human body is subjected to certain environmental 
conditions such as extreme cold, the pituitary responds by 
secreting ACTH, 

At a result the quantity of ____ * cr *ted by 

the adrenal cortex ( i n c reases/de c r eases) ^^ . 



366 



APPENDIX I 



auouuoi) 



jo UOJ43JD3S 541 BuisoaJouj Xq spuodsaj X34JOD jouajpo ; 



*09 



XjD4in4jd 3 
psuiol s; 




*MO|tC| LUDJ6oip 3^4 Ul pUD|6 

*>||D4S D Xq uiojq 344 40 spisaspun ai|4 04 
D jo 9zis ai|4 4noqo si puo|6 XjD4]n4jd aij^ 




Mopq UMOLJS puo|6 puajpD 944 jo oiDj6Djp sij4 |aqo"i 



XS4J03 
XS4JOD 



APPENDIX I 



367 



Answer to Question I: 
I. gland 



Organs that secrete substances affecting processes in 
various parts of the body are called 



; TURN PAGE FOR ANSWER AND QUESTION 3. ', 



22. 



21. cortices 



The steroids are o class of 



secreted 



only by the tesres, ovaries, placenta and adrenal cortices. 



42. 



41. brain 



Below is a diagram of the underside of the brain. 
Shade in the pituitary. 

FRONT 



BACK 




62. 



61. corttcosteroids; 
increases 



An increase in the secretion of ACTH by the 
result's in a(n) (increase/decrease) ___^__ H ___ 
corticosteroid secretion by the adrenal 



gland 



(which part?). 
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2. glands 



Glands that secrete substances directly into the bloodstream 
are endocrine ( en' do crTne ) glands. The secretions 

of glands travel in the bloodstream to 

various parts of the body. 



23. 



22. hormones 



The adrenal cortices, testes, ovaries and placenta secrete 
the class of hormones known as the 



42. 



FRONT 



43. 




The pituitary produces a hormone which controls the 
secretion of most of the hormones from the adrenal 
cortex. In other words, the amount of 



(type of hormones) found in the bloodstream Is largely 
dependent upon a hormone from the pituitary gland. 



BACK 



63. 



62. pituitary; 
increase; 
cortex or cortices 



A large amount of corticosteroids in the bloodstream 

inhibits the secretion of ACTH. 

After the hormone .______________ nas stimulated 

the secretion of corticosteroids, the corticosteroids 
travel in the bloodstream to the pituitary where they 
(increase/decrease) _____ > _ > _ - ^ --- _____ > _ i * n rafe f 

ACTH secretion. 
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3. endocrine 



The thyroid gland secretes a hormone called rhyroxin 
into the bloodstream. The ovary *ecretes a hormone 
called estrogen Into the bloodstream. In general, the 
substances that are secreted into the 
by endocrine glands are called 



24. 



23. steroids 



How many different sets of steroid-producing glands 
ore there in the body of the normal male? 



44. 



43. cortlcosteroids 



The ovaries, testes, thyroid and adrenal cortices 
are target organs of the pituitary gland. In general , 
we say that an organ li a ^ 



stimulates that organ . 



of the pituitary if the pituitary 



64. 



63. ACTH; 

decrease 



ACTH (stimulates/inhibits) 



the 



secretion of corticosteroids, whereas corticosteroids 

(stimulate/inhibit) the secretion of 

ACTH. 
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4. bloodstream; 
hormones 



The pituitary gland secretes hormones into the bloodstream. 
Therefore, we call the pituitary an endocrine gland. The 
adrenal glands are also endocrine glands. Why? 



25. 



24. two (2) 

(testes and adrenal 
cortices) 



The ovaries produce primarily a certain kind of steroid 
called estrogen. The testes produce primarily a second kind 
of steroid called androgen. The placenta produces mainly 
the kind of steroid called progesterone. The adrenal cortices 
produce a fourth kind of steroids called corticosterolds. The 

steroid producing glands produce (the same/different) 

steroids. 



45. 



44, target organ 



Only one part of the adrenals is affected by the pituitary. 
Name the part which Is not affected by the pituitary. 



65. 



64. stimulates; 
Inhibit 



The relationship between the pituitary and the adrenal 
cortices is a two way relationship. Th hormone 

stimulates the secretion of corticosteroids 

and these in turn (inhibit/stimulate) 

the secretion of this pituitary hormone. 
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5 because they secrete 
hormones into the 
bl oodstream 



6. 



Hormones are substances secreted into the 

by . " 



25. different 



26. 



The steroids produced by the adrenal cortex are 
called cortico 



45, medulla or 

adrenal medulla 



46. 



The adrenal cortex is one of the 
organs of the pituitary. 



65. ACTH; 
Inhibit 



66. 



If a doctor were to infect corttcosteroids into the 
bloodstream, what effect would this have on the 
secretion of ACTH by the pituitary? 
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6. bloodstream; 

endocrine glands 





'food vessels 



Certain glands, such as the salivary glands, secrete 
substances (not hormones) into ducts. Endocrine glands, 
on the other hand, are not connected to ducts; they 
secrete their hormones directly into the bloods frea 
Endocrine glands are less glands. 



27. 



26. cortlcosteroids 



The 

the adrenal 



steroids ore hormones secreted by 
(which part?). 



47. 



46. target 



The 



one of which stimulates the adrenal 
secrete 



_ gland secretes several hormones, 



67, 



66. The secretion of 
ACTH would be 
inhibited. 



ACTH (stimulates/Inhibits) the secretion 

of / whereas cortlcosteroids 

(stimulate/inhibit) __ the secretion 

of 
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7. ductless 



GLAND DUCT 




Why Is the gland shown above probably not an endocrine 
gland? 



28. 



27. cortiosteroids; 
cortex (cortices) 



The adrenal cortex secretes several hormones into the 
bloodstream. Collectively these hormones are 
called 



48. 



47. pituitary; 
cortex; 
corticosteroids 




A. 

The pituitary hormone which affects the adrenal cortex 
helps to maintain the size of the cortex by stimulating 
the production of corticosteroids. Which of the above 
adrenal glands was not affected by a sufficient amount 
of the pituitary hormone? (A or B) 



68. 



67. stimulates; 

cortlcosterotd* 

Inhibit; 

ACTH 



Explain the two-way relationship between ACTH and the 
corticosteroids. 
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8. Endocrine glands 
are not connected 
to ducts. 

(or equivalent 
answer) 


9. 

Substances secreted by duct glands are not called 
t whereas substances secreted by 


endocrine glands are called 




28. corHcosteroids 


29. 

The part of the adrenal gland that does not secrete 
corticosteroids secretes two important hormones 
called adrenalin ( 6 dre'na lin ) and 
noradrenalin ( nor' a dre"'na lin ). The adrenal 
medulla secretes and 




48. B 


49, 

The suffix "trophic" means "nourishing". Thus, if a 
hormone is described as trophic, it es 


or stimulates its target organ. 


68. ACTH stimulates the 
secretion of cortico- 
steroids while 
corticosteroids 
inhibit the secretion 
of ACTH, 

(or equivalent 
answer) 


69. 

The word "atrophic" means almost the opposite of trophic. 
Whereas trophic means nourishing or stimulating, "atrophic" 
means a wasting away of an organ. Thus when an organ Is 
described as atrophic, it is under 
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9. hormones; 
hormones 



10. 



Hormones are secreted by 



glands. 



TURN BOOK OVER TO PAGE I! FOR ANSWER TO QUESTION 
10 AND PROCEED TO QUESTION II. 



30. 



29. adrenalin; 
noradrenalin 



The name adrenalin is easy to remember because it consists of 
the two words "adrenal" and "in" , and because adrenalin is 

one of the hormones secreted by the side part of 

the gland, which is the medulla. 



TURN BOOK OVER TO PAGE II FOR ANSWER TO QUESTION 
30 AND PROCEED TO QUESTION 31. 



49. nourishes 



50. One of the hormones secreted by the pituitary is called 
adrenocorticotrophic hormone. 



adreno refers to the 
corttco refers to the 



gland, 
of the 



gland. 



means nourishing or stimulating, 

TURN BOOK OVER TO PAGE II FOR ANSWER TO QUESTION 
50 AND PROCEED TO QUESTION 51. 



70. 



69. 



undernourished or 
undersfimulated 



If the pituitary is destroyed or removed, the cortices of both 

adrenals are no longer stimulated by the hormone . 

As a result the cortices become thin and undernourished. In 

other words, the adrenal cortices become in the 

absence of the pituitary. 

TURN BOOK OVER TO PAGE II FOR ANSWER TO QUESTION 
70 AND PROCEED TO QUESTION 71. 
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TOR 

EDUCATIONAL PRODUCT 



MEMOREASE 



Distributed by 



TOR EDUCATION INC. 
55 Fifth Ave. 
New York, New York 



A MEMOREASE 

This is a fast, easy method of memorizing 
a poem. If you follow the directions, you 
should memorize this poem in about ten 
minutes. 

In this poem, Miss Dickinson is expressing 
wonder as to the nature of immortality. There 
have been two events in her life which pre- 
sented a foretaste of dying Can the third 
event (death) compare with those two? Part- 
ing is, after all, all we know about death 



MY LIFE CLOSED TWICE, 
BEFORE ITS CLOSE 
by Emily Dickinson 

"My life closed twice before its close, 
It yet remains to see 
If Immortality unveil 
A third event to me, 
So huge, so hopeless to conceive, 
As these that twice befell. 
Parting is all we know of heaven 
And all we need of hell." 



Directions 

Read the first exercise on the pleated sheet, 
Orally supply the missing word or words. 
-Fold the first pleat back and read the 
second, filling m all missing words. 

Continue until you are at the bottom of the 
page. UnfoJd the pleats and continue on 
the back side. 

Complete the poem each time. Look back 
one exercise ii you should forget something, 

NOW, FOLD THIS PAGE BACK 

O 1961 by TDK Education Inc 
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So hii{<e, so hopeless to conceive, 
As thet.0 that twice befell 

Partmi* is all we know of heaven 
And all we need ot ___. 



So hm;c, .so hopeless to conceive 
Ah these that Hvire befell 

Parting is all . 
And all - 



So hu.i^e, so hopeless to conceive, 
As these that twice . 

Parting : , 



huge, so hopeless to 

As that twice . 

Parting , 



If Immortality unveil 
A third event to rne, 

huge, so hopeless to 

that . 



If Immortality unveil 

A third event to me, 
. huge, so hopeless to - 

that . 



My life closed twice before its e 

'it yet remains to see 
If Immortality _ 

A ............ _________ _____ to rne, 



_ that . 



, 
. hopeless . 



My life closed twice before its , 
It jet to see 

If Immortality 



My life twice its - 

yet to see 

If Immortality I1 .. J . I 



My life _ 



_ yet _ 



My, 



My __ 
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